Hello again! We have two
devlog updates that have happened within the past few days, so I decided to post them both at the same time. The first one is from Lindsey, and the second one is from Josh. Enjoy!
Friday, December 1, 2017
Originally posted by LindseyReid
First of all, thank you for everybody (espeically our potentially confused non-American forum-goers :V) for being patient while we all enjoyed our Thanksgiving break ^^ Since more holidays are coming up again soon, I'll be posting one more devlog this year after this post. My last devlog of 2017 will be two weeks from today, on Friday, December 15th.
After that, my devlogs will resume again on a normal two-week schedule on Friday, January 12th
Josh and Adam don't have exact devlog plans like this, but I would expect them to be similarly interrupted by the holidays. Although I can
tell you that y'all will probably be quite pleased by the progress they've been making during this devlog silence. ;00 We always appreciate your patience <3
If you need a daily shot of LT, you can follow me on Twitter
, where I post screenshots almost daily. In addition I'll also link every LT tweet on these devlog threads.
That way, those of you who don't have a Twitter can check back here for those screenshots
Although the tweets will be more frequent than devlogs, expect my LT-related tweeting to be sparse around Christmas through New Year's Eve.
Speaking of screenshots...
Limit Theory Devlog Gallery - Lindsey 12.01.17
Now, on with the words!!!
11.10.17 - 12.01.17
- Added more data to Joints
- Added up vector
- Added scale
- Added function to generate a JointField from a Shape
- Added AABB function to Shape
- Cleaned up all basic Shape functions
- Ensured all shapes are manifold
- Ensured no basic shape functions create bad polys
- NEW SHAPE: 5-faced pyramid
- NEW WARP: Bevel
- NEW WARP: Smarter tessellation
- Refactored tessellate() to tesselate quads INTO quads
- Refactored tessellate() to produce 0 duplicate verticies
- NEW WARP: Greeble!!!
- Implemented a policy for protecting against crashes & bad geometry as a result of trying to apply a warp on a bad poly.
- Began work on Style
Boring Architecture Stuff
Joints, as y'all remember, are used to define patterns of shapes and create clusters of shapes. I added two crucial pieces of data to this to really bring out the strength of Joints: a Scale and an Up vector. Joints now store Position, Direction, Up, and Scale. (If you're familiar with Unity, they're basically Transforms at this point that don't have any other components like a mesh.) The Scale allows the user to define how a Shape will grow or shrink in size when placed on the Joint
. The Up vector is a mathy thing that gives more strength (and prevents errors) when using the Direction property to define what direction the shape will face in when placed on the Joint. To complement this, I made sure that all of the basic Shape creation functions (like ellipsoid, box, and icosahedron) as plain as possible. They just give you a shape, and require no input regarding rotation, translation, or scale, as all of that is handled by Joints now. I also ensured that all basic shape functions return a shape that fits inside a (1,1,1) box, and added an AABB function to get size information about shapes. All of this work combined means that we now have more intelligent ways to combine shapes based not only on some abstract position, but also on a position relative to a shape's scale
Here's a small demonstration of AABBs & the new scaling system working. The scaffolding blocks to the left of the sphere are using the sphere's AABB information to be placed perfectly at the edge of the sphere and span the length of the sphere.
Here are two demonstrations of the new Joint properties. Both images use the function to generate JointFields from a shape; they generated a Joint for every quad on the large box, then added smaller boxes to every one of those joints. The second one demonstrates using the Joint's new Scale property to scale the size of each smaller box based on its Y-location on the larger box.
I also spent a day ensuring that every basic shape function produced perfect data
. Specifically, I eliminated any 'bad' polys
and ensured all of the shapes are manifold
. (A 'bad' poly is defined here as having a zero-length normal and/or fewer than 3 verticies.) The Irregular Prism was creating one bad poly, so I got rid of it. The Box was non-manifold, so I re-wrote it to be manifold. For the purposes of fixing the Box, the mesh being 'manifold' means that there are no duplicate verticies with the same location
. Put another way, two indicies in a poly may refer to the same vertex, but all verticies with a unique position have only one listing in the Vertex list. This isn't the complete definition of manifold, just to be clear!!!, but it worked for the purposes of fixing the Box algorithm.
After ensuring that no basic shapes generated bad data, I implemented a new policy for protecting against attempting to warp invalid polys.
This is a bit of a tricky situation, because while I can guaruntee that the original input to a slew of warping functions is perfect, I fear that our infinite universe with its random combination of warps could still create bad polys. For example, tessellations greatly reduce the size of each poly every time they're applied, since you're splitting one poly into multiple pieces. This could theoretically create a situation where a triangle became narrow enough that it was essentially a line, and therefore had a 0-length normal. So, for the purpose of preventing crashes & junk shapes in release, we SKIP performing any operations on invalid polys.
From the testing I did, this 1) creates shapes that only have 'good' polys operated on, which still look good, and 2) prevents potential crashes and ugliness from the spreading of NANs and other unwanted mesh data. I'm going to continue thinking about this problem further, as it's a rather dry, but still suuper-important one in the world of limitless procedural geometry.
Before the break, I had begun working on Style. Style is the object to generate and store all of the properties that make a faction's aesthetic unique.
Honestly, I'm not super happy with how my first implementation of Style worked out. I think my biggest struggle with it is that I don't have a good foundation to test it on yet.
It'll be more useful to start iterating on Style when I have at least one complete station part or one complete ship type to work on. Before Thanksgiving break, I had been working on building station parts from a perspective of starting with extreme randomness, and THEN trying to refine it. Josh had also previously tried this method, and we both found it to be ineffective. Instead, next time I attempt building functional-looking parts, I'm going to start with a low randomness/ high functionality algorithm and THEN stretch the algorithms until I reach the limits of what still looks recognizable.
I believe that I'll have much better results with both making recognizable parts AND iterating on the Style architecture if I come from that perspective.
Fun Shapes & Warp Stuff
I added a 5-faced pyramid shape.
I also contemplated n-faced pyramids, which would also allow cones, but then got distracted by something else I think. Now that I'm here writing about it, I'll probably do that tonight & update y'all later. LOL.
Josh, in his spare time, wrote a bevel
function, because, of course, Josh. It's quite incredible- it can take ANY shape and round ALL of the edges by an arbitrary amount. The 'amount' is a number between 0-1 which describes how rounded the corners will be- 0 leaves them sharp, and 1 rounds them all the way to the next edge.
Here's a bevel on a basic cube.
The same cube, with a higher amount of bevel.
The same cube, with a bevel amount of 1.0.
A bevel on a cube whose faces have been extruded.
A prism that's been beveled, extruded, and then beveled again.
I added a few different variants on tesselation, and generally made the algorithms & their uses smarter. We now have several options for tesselation & triangulation
- A fan-shaped triangulation, which creates 0 new verticies. This doesn't preserve the angles of the original poly, and thus is currently only applied at the last step of shape creation, right before the shape is converted into an engine-level mesh, which is used for rendering.
- A centroid-based triangulation, which doesn't preserve angles, and is used to break up polys with >3 indicies.
- A triforce-shaped triangulation, which preserves the angles of the original poly. Currently, it can only be applied to tris, so right now a centroid-based triangulation is applied to any polys with >3 indicies before applying the triforce triangulation. This is great for creating detail is meshes before applying warps like greeble().
- A tesselation for quads that splits them into smaller quads. A quad is a poly with 4 indicies- basically a rectangle or square. Tessellating quads into more quads preserves the original intention of the poly better than splitting it into tris. It's used in our shape:tessellate() algorithm in conjuction with the triforce-shaped triangulation to create more detail on shapes before applying ceratin warps.
Here's a hand-drawn picture from yours truly to show what the different triangulations look like:
Aaaaand finally, I wrote a Greeble function!!! w00t!!! Greebles are small, repeated details added to the surface of a mesh to give it the appearance of detail and scale. For a classic example, think Star Wars ships:
Here's a box, in LT, beveled, extruded, then greebled:
And here's a long, greebled tunnel:
My plan moving forward is quite exciting: between now and the new year, I want to jam on generating a complete bomber-sized ship
. I feel pretty good about the library of shapes, warps, and tools to use and combine them that I have built up. Previously, I wanted to work on station parts first, but I hit a wall with them because of their complexity. Station parts need to look extremely functional, which leaves little room for experimentation and a lot of room to get frustrated about whether something looks 'real' enough or not. I can leave that frustration for later, once I have a grasp on generating something small & have built a more robust Style system. A small ship would just be more exciting and a bit simpler. In addition, this would give me a more solid base to work on to iterate on the Style implementation.
Expect exciting devlogs coming up! Thank y'all so much for joining me in the 3rd devlog of my LT journey. Remember that I've got one last devlog after this until 2018! But you can always catch me on the forums, Twitter, or the IRC in the meantime
Gallery link again, because DUH.
P.S.: for anybody who enjoys my blog tutorials - I WILL be writing a greebling tutorial & a triangulation tutorial soon! Let me know in the comments which of those or other topics you'd like to see.
Link to original: viewtopic.php?f=30&t=6380
Monday, December 4, 2017
Originally posted by Josh Parnell
Hey again everyone! Hope you all had a good Thanksgiving (for those to whom it is applicable). I certainly ate my fair share
It's been a productive few weeks since the last log (minus Thanksgiving break), but last week, in particular, saw two major
breakthroughs that have me really happy with the current state of affairs
Major Progress Since Last Time:
- Major breakthrough with being able to use Lua data & functions from our C subsystems
- Used the above to implement a C-side scheduler that schedules Lua logic for execution; significantly faster than previous implementation
- Ported planets & atmospheres
- Integrated Loom for advanced inspection of LuaJIT traces and assembly
- Implemented several new algorithms for procedural geometry library, touched on in Lindsey's latest log
- Loads of quality-of-life improvements to how 'game-level' Lua is written
- First-pass design of mod format taking into account all existing support systems
We're All Modders: Hammering out LT Mod Format for Building Gameplay
At this point, we've essentially got all of our core support systems working. By that, I mean that we have either Lua-level or C-level (in some cases a mixture) mechanisms for supporting almost everything gameplay code could want to do: from defining new datatypes that can be used with the utmost efficiency, to adding and/or modifying existing ones, to writing gameplay code that needs to run in response to certain events, or needs to be scheduled to run at certain frequencies, to writing new methods that can be called on the aforementioned datatypes, to defining jobs that batch-process large chunks of entities that match a certain filter, and so on. Where there was once a wild-west of Lua trickery, there is now a solid set of production-ready, battle-tested system. So, what remains? Naught but the orchestration of said systems to compose the symphony of gameplay that is Limit Theory
So, it is time that we once again turn our attention to the format
for this orchestration: the Limit Theory Mod Format. I have expressed many times that I wish for our vanilla LT gameplay-level logic to be written in exactly the same manner as any other mod, and I'd like to stand by that -- both for the sake of making it easier on us devs (since we'll have a well-defined, decentralized framework for incrementally adding modular bits to the game), as well as for the sake of ensuring that LT is not just moddable in theory
, but is built
on the very proof that it's moddable in practice
Thus far, my solution is unfolding as a two-part mod format, where each part may exist zero or more times within any given mod file. One part is data, the other part is code. Data sections use a very simple 'data description language' to perform the aforementioned orchestration of our system. A data section for implementing an inventory might look like this:
Hopefully it doesn't take too much brainpower to figure out what's going on here: we've opened up some object scope called 'Ship,' declared that it has an Inventory, and then gone on to define Inventory itself. Inside Inventory, we can see three types of 'declarations,' each of which corresponds concretely to a programming concept and each of which will be used in a different way to feed our underlying systems to build the required functionality.
As the programmers among you can likely guess, 'has' declares a field/member variable, 'can' declares a member function, and 'when' declares an event listener (a piece of code that will run in response to a particular 'event' that may occur on the object). The syntax of it all is just a sketch, so let's not concern ourselves too much with quibbling over whether 'has/can/when' is too cute / whether semicolons are the bane of mankind, etc
The point is that we have a portion of the mod that's using a very simple 'data' language to inform all of our heavy-duty systems of how to make inventories work
. The 'has' statements inform our CTypes system about fields, address, alignments, etc. The 'can' is informing our more-general type system about the presence of callable member functions. The 'when's are informing our event handler system about events and their listeners.
Then, of course, we must write some Lua to make it all work:
And this looks more like what you would expect from standard Lua code. Here we're just implementing the functions referenced in the data section with standard Lua + some of the extra features added by the LT engine. Easy! All the 'bigger picture' elements that dictate when, where, and how a particular function runs are specified in the data section (e.g. addItem will be a method that can be called on any object with 'has Inventory', so that in other code we may say myShip:addItem(DenseTalvienite, 17) for example). Hence, much of the need for a modder to 'worry about' the internal mechanisms used to make LT performant (you know, the ones I talk about in pretty much every devlog ever?) is alleviated. Granted, they still exist and may still be interacted with at the code level, but for the majority of gameplay functionality, doing so should be unnecessary.
The Lua mod loader that parses all the mods into a format that can then be shipped off to the appropriate engine-level systems is already implemented
; what remains is to do the shipping-off bit and to move existing gameplay code into isolated mod files where appropriate. I'd like to have this done by the end of this week
In response to anticipated concerns about 'oh no another language / LTSL / JOSH BE CAREFUL'
: Rest assured, the goal is entirely different here. I didn't define a new scripting language; this is just a very simple
data-description language that helps us connect all the moving pieces of our mod! It can be parsed with ~20 lines of Lua and the lovely string.gsub function. Anything more complex is unnecessary. I am not rabbitholing, I promise
Side-Soliloquy: Why So Much Concern for Moddability?
This bonus Josh-soliloquy is an optional part of your devlog experience
Many of my technical expositions, and even the less-technical ones, are ridden with talk of moddability and concern that a given system will not hinder the extensibility of LT. I feel that it might be a good time to step back and look at the big-picture of LT development for a moment; to express what modding means to me, what it has meant for Limit Theory's development process, and how it has impacted both to the utmost degree. To open, I'm going to make a daring assertion: I believe that Limit Theory would already have been released by this point had I chosen to stick with my original vision of 'no modding.'
Take it as you will, perhaps noting that I do have a patently lackluster track record of timeline judgement (:oops:) Still, I feel that there is quite some truth to the statement. After all, we had the solid beginnings of a game with the release of the LT Prototype all the way back in April 2013
Sometime in 2014, the tides began to change. I began to recognize the need for an external data format in which to place 'arbitrary' constants like those relating to high-level balancing (how much energy is required to negate one unit of damage compared to dealing one unit of damage? How much energy is required to extract one energy unit's worth of raw material from an asteroid? etc..) Naturally, it wasn't long before a similar need for 'arbitrary' code was recognized. The solution at the time was LTSL, and, as we all know, thus began the march toward the dark days, FPLT, and some of the hardest months of my life.
Looking back, it's clear that my solutions, from that point on, were guided by an unspoken requirement: LT had
to be moddable, on that point I was no longer willing to yield -- else I would surely have eventually come back to C/C++, learned to write a more modular engine with faster compile times, and found a way to iterate on gameplay code without needing a separate language. It's hard to admit. But it's the truth. Hence, today, we are in a world where great sacrifices in development time and effort have been spent on Limit Theory's touted modding capabilities. Personally, I'm OK with that. And here's why.
When others ask about LT, moddability often comes up. Invariably, I bring up Limit Theory's suggestions
sub-forums at some point, typically bragging on it as being "surely on of the best places on the internet to find inspiring, space-game/sci-fi ideas of remarkable breadth and depth, elaborated on by people of singular imagination and intellect." Since the KS closed at the end of 2012, there have been >1,000 topics and >20,000 posts in this subforum alone. It is a remarkable place -- one that I wish I had the time to read front-to-back. But that's just it: I don't. I don't even have enough time to read
all of the beautiful, elaborately-thought-out gameplay suggestions generated by you all, much less implement
What I can
do? What I'm really good at doing
, as this whole process has taught me, and perhaps, what it is that I love doing above all else
: it's probably not game design, it's probably not even procedural nebula algorithms
...no...building things that allow other people to realize creation with ease, simplicity, and power...now that
is the kind of thing that can get me out of bed in the morning. And that's exactly what I/we have done. We've built something that can take a high-level, simple description of a new gameplay mechanism, and do everything that needs to be done behind-the-scenes to let that mechanism play nicely with a thousand others, in the presences of tens of thousands of entities stretched accross a boundless universe. We've built something that renders many of your suggestions doable in a few hundred lines of Lua or less. And THAT...I'm proud of that
Everyone knows that Morrowind was one of my favorite games of all time. And it still is. It's still cranking, 15 years after release (in 2002). People still play it, people still buy it, people still love it. All because Bethesda decided they would build their game with a Construction Set
rather than pure code, thereby enabling others to add
to their original vision (which was already fantastic) via that tool. And the result is that today, some of us are still talking about how stunning Morrowind looks with graphics overhaul 3.0 et al; some of us are still in awe that Tamriel Rebuilt is alive-and-kicking, expanding the vanilla game's playable area by god-only-knows how many square miles.
And this is my dream for Limit Theory: that it ships as the vision you all supported in 2012 + the modding capabilities that have since become a core feature, and that, in 10 years' time, it has become something utterly and totally beyond that which I could have done myself or with our small team in the same amount of time. I really can't wait
I've left out all talk of a 'breakthrough' that happened last week involving Lua data playing nicely with our C systems. It's technical and I've decided to err on the side of not-so-technical this week, but, briefly, I'll just say that this breakthrough has opened up a lot
of fantastic augmentations to our existing systems, as well as tremendous new flexibility in splitting data & logic between Lua and C (using whichever makes the most sense for the task at hand). Previously, Lua->C interop was easy. LuaJIT's FFI makes it basically effortless. But going the other way? Accessing Lua-side data / calling Lua-side functions from C? It was not so easy when that data/function could be arbitrary and
needed to live alongside supporting C data (think: sparse logic scheduler). Since Lua is GC'd, we can't get pointers to Lua data, and this is a major annoyance about which I've ranted before. However, there's a function that I overlooked until last week...hidden in the Auxiliary Library API...luaL_ref...and it solves all my problems. *Slams face vigorously for having overlooked this gem.* At least I know now.
Until the 'morrow! (Well, actually I plan to do my next log on Friday. If I'm 'close but not quite' with respect to the mod system, then it will be Monday. Stay tuned!)
PS ~ Sorry for lack of screenies, but most of this work wasn't amenable to it. Besides, you guys just got a whole boatload of them in Lindsey's last log!
Link to original: viewtopic.php?f=30&t=6385