Hey all!
As usual, I apologize that these logs are doing a bit of temporal stretching lately -- honestly we're just really, really busy working on LT and it's tough to pull myself away these days, so consider it a good thing! When we last spoke, several major advances had set us up for fruitful weeks to come. I'm pleased to report that they were, indeed, very fruitful.
At the moment, I can summarize our work by saying: we're dangerously close to shifting the majority of our efforts to sprinting on LT gameplay code in Lua. And we're doing everything we can to close the gap.
Seriously, the 'road to (performant) gameplay' has been way longer than anyone wanted. But the end is in sight
---
Josh
- Finished implementing zero-cost Lua wrappers for engine functionality (mentioned in last devlog; means we're getting superb performance when interacting with the LT Core)
- Implemented enough of the foundational Lua code that we can all work on LT in Lua at the same time
- Continued careful analysis of LuaJIT debug output; continued to ensure 100% JITing to assembly; continued to learn more about the trace compiler and how to beat it into submission
- Hooked up our ECS to Lua + wrote the convenience wrappers that allow it to be used really easily (note: we now call it the 'CType' system instead of 'ECS', reason being that it is actually more general and flexible than an entity component system, although it can and does serve as one -- I'll explain more below!)
- Ported Lua deferred renderer; modified to take advantage of all high-performance constructs
- Made major improvements to our logarithmic z-buffer code while doing the above (this is the code that allows us to render objects at massive scales & distances without issue)
- Implemented a work-in-progress event system in Lua for use in gameplay code
- Implemented Lua wrappers for efficiently dealing with SDFs and OpenVDB (the library mentioned in the last log that we now use for extracting meshes from distance fields)
- Implemented CSG functions on SDFs that are roughly 10x faster than OpenVDB's defaults, allowing complex, detailed meshes to be built very quickly (read: better procedural mesh algorithms coming soon!)
- Wrote a simple pathtracer for fun since graphics Josh has been repressed Not totally irrelevant to LT, though, as we could perhaps use it for nice renders of procedural assets or for having a 'ground truth' reference for our physically-based rendering code
- Built several Lua 'template' applications that allow us to easily various, specific pieces of functionality really quickly (just like the old LTSL sandboxes!)
- Loads of other things that I'm forgetting
Adam
- Started working in the Lua side of the LT codebase!
- Put in the time to learn about all of LuaJIT's lovely 'idiosyncrasies'; Adam is now 100% capable of helping me deal with the technical side of LJ
- Used LuaJIT magic to implement the first parametric type in our CType system -- the arraylist! This means that not only can we define and use new, native datatypes in Lua, but we can also have what amounts to native std::vector<T> and so forth of them. I know I've beat this horse to the ground, but seriously, the power we now have to create and use types that are as fast as compiled code simply can't be overstated! Memory management is by far the weakest spot in Lua's performance, and we are almost entirely side-stepping it.
- Hooked up BSP code to Lua and used it to implement picking (being able to click on an object in the world and do something in response -- a seemingly-trivial task that is not at all trivial for large scenes and complex meshes, since it requires casting a ray through acceleration structures)
- Got our first star system set up and working using CTypes and all the new high-perf Lua stuff!
- Made a lot of improvements to our geometry code (BSPs, triangles, rays, etc.)
- Implemented really fast BSP-Sphere intersection tests
- Loads of other things that I'm forgetting (again)
Sean
Sadly, last Friday was Sean's final day here as an intern, since he's now headed off to college to study CS and become an unstoppable force of programming. We'll miss him for sure...his time here seemed way too short. Nonetheless, he was able to accomplish an impressive amount in the seemingly-little time that he had.
In his final act of glory, Sean finished implementing a fully-generic, hierarchical, fleet-based AI maneuvering system, capable of representing fleets with any number of sub-fleets, and intelligently directing their motion. To demonstrate it, he showed us a group of several fleets flying together in a V formation, with each fleet comprised of squadrons in a V formation, each of which was comprised of fighters in a V formation...you get the idea! All of the math required to make it happen is rather difficult due to this arbitrary nesting requiring scale-invariance. Sean got it all hammered out and now we have proper, hierarchical fleets. I'm really excited to see this in-game I don't have a gif on-hand, but I'll be sure to show off his work as soon as we attach it to the main LT Lua code.
---
High-Performance, High-Convenience: A Peak at Coding in LT
As you all know, Adam and I have worked relentlessly over the past months to devise ways to achieve both extreme performance capable of handling the LT simulation logic, as well as the convenience of being able to write said logic in a language that won't rob us of every last drop of life. I'd like to show just a snippet of code demonstrating what our native type system ('CType' -- the final version of what we used to call the ECS) can do, how it looks, etc. So, have a slice of toy code! (Since it's harder to read without syntax highlighting, I just screenshotted some code in-editor)
This is a massively-simplified battle simulator, intended only to showcase the simplicity of gameplay code. Nonetheless, it demonstrates some important features / fruits of our labor. Note: I cranked this out really quickly and didn't actually test it, there may be typos or slight mistakes )
Notable Take-Aways:
- Native memory types can be defined on-the-fly in Lua; behind the scenes they are 100% as compact as the equivalent type would be in statically-compiled C; accessing the memory is extremely fast due to fancy, custom memory-pool allocation magic done automatically by the engine. All of this complexity is hidden from the user, so code remains very easy to read and write.
- Types can be nested in other types, enabling 'component-based' design, as demonstrated with Health and Weapon. Of course, since this is a full-blown type system rather than an ECS alone, we can do things like have multiple health components, which ends up being useful sometimes!
- 'Member functions' can be attached to these types and called in the idiomatic Lua manner; convenience + performance = winning!
- Generic functions like 'Weapon:damage' can be written without concern for the type of the target object and still 'just work,' despite the fact that we're using direct field access instead of table lookups like most dynamic languages (if you're a programmer and understand what I mean, that should blow your mind a bit )
- (Here's the even-more-mind-blowing magic) Functions that can operate on different types of objects (like Weapon:damage, which could be used to damage stations, hardpoints, etc) will be traced by LuaJIT and end up running as fast as equivalent statically-compiled constructs (in C this would require a switch statement or function pointer, in C++ this would require a template or virtual tables; here it requires nothing extra!!); this is a HUGE win for both performance and simplicity!
- Although I didn't show it above, types can be built in a piece-meal fashion rather than defined all at once. In particular, mods have access to type definitions and can inject their own fields and methods as required to support extra, mod-related data and functionality (I showed something like this in a devlog a while back). As always, giving modders the ability to add just about anything conceivable to vanilla LT is really important to me!
---
Well, once again I've completely and utterly failed to be brief It's hard to be brief when there's so much happening and so much about which to be enthused. Speaking of being enthused. I must leave you all now to return to the code
Hello, everyone! Talvieno here with a non-technical summary. There hasn't been much demand for one for a while, but this devlog seems to be worthy of it. Without further ado, let's get down to business! There are a couple "big things" to take away from this update - namely, Sean is no longer with us, and we're almost ready to start focusing entirely on gameplay. As to Josh's bullet lists, I'll break down what those mean in the spoilers below.
Josh
(tl;dr: made things much faster and more performant)
Spoiler: SHOW
Adam
(tl;dr: Lots of coding wizardry)
Spoiler: SHOW
As to all the last part (the image, the code, the final bulletpoint)... that can be summarized by saying Josh is proudly showing off the fruits of his labor, which unfortunately may not mean much to anyone that isn't a well-established programmer that can at least code on an intermediate level. The main things to take away from it is that it's fast, and flexible, meaning it's the perfect tool for modders. I'm actually understating it here - this is some top-tier wizardry that Josh and Adam have written up. If you're used to games running considerably more slowly when you start adding mods, that shouldn't be the case with LT. For those interested in writing their own mods, all this also means that you'll have a ridiculous amount of control over the functionality you can add to the game.
And, that's it! Josh failed epically at being brief (does he ever not? We love him for it), but I've hopefully condensed it down to something a bit more pleasant to read for the non-technically inclined. I'll answer any questions as necessary, and thanks for reading!