Return to “Announcements”

Post

Re: The Limit Theory Email/RSS News Thread

#31
I've decided to include a small post by AdamByrd (one of the two coders Josh hired as help) as a temporary stand-in while we wait for an update from Josh himself. :)
AdamByrd wrote:
Tue Aug 15, 2017 9:59 am
Guys! Gals! Hey. I'm just popping on to check in with a super ultra tiny mini update.

Things are starting to move fast. Lil'T is alive and kicking. And producing pixels. Actual pixels! (Seriously, I've spent so much time staring at text books and data dumps these last three months I almost forgot we're making a game. With nebulas and rocks and space pirates.) I've been working to nail down how we're going to be writing actual gameplay code. That means fleshing out how Lua talks to C to keep heavy bits and allocations in fast code land, how to make that Lua feel 'normal' and not like C grafted onto Lua (metatables are really nice!), making sure LuaJIT actually jits to assembly, iterating on the engine API to remove friction, and building a prototype solar system. I'm insanely excited about the progress we've been making. All the mountainous unknowns are now small piles of exploded rubble. We're moving faster by the day.

The GIF floodgates are opening soon, my friends.

What a time to be alive.


Link to original: viewtopic.php?f=30&p=153664#p153643
Have a question? Send me a PM! || I have a Patreon page up for REKT now! || People talking in IRC over the past two hours: Image
Image
Image
Post

Re: The Limit Theory Email/RSS News Thread

#32
Friday, August 18, 2017

Hey all!

As usual, I apologize that these logs are doing a bit of temporal stretching lately -- honestly we're just really, really busy working on LT and it's tough to pull myself away these days, so consider it a good thing! When we last spoke, several major advances had set us up for fruitful weeks to come. I'm pleased to report that they were, indeed, very fruitful.

At the moment, I can summarize our work by saying: we're dangerously close to shifting the majority of our efforts to sprinting on LT gameplay code in Lua. And we're doing everything we can to close the gap.

Seriously, the 'road to (performant) gameplay' has been way longer than anyone wanted. But the end is in sight :)

---

Josh

  • Finished implementing zero-cost Lua wrappers for engine functionality (mentioned in last devlog; means we're getting superb performance when interacting with the LT Core)
  • Implemented enough of the foundational Lua code that we can all work on LT in Lua at the same time
  • Continued careful analysis of LuaJIT debug output; continued to ensure 100% JITing to assembly; continued to learn more about the trace compiler and how to beat it into submission :ghost:
  • Hooked up our ECS to Lua + wrote the convenience wrappers that allow it to be used really easily (note: we now call it the 'CType' system instead of 'ECS', reason being that it is actually more general and flexible than an entity component system, although it can and does serve as one -- I'll explain more below!)
  • Ported Lua deferred renderer; modified to take advantage of all high-performance constructs
  • Made major improvements to our logarithmic z-buffer code while doing the above (this is the code that allows us to render objects at massive scales & distances without issue)
  • Implemented a work-in-progress event system in Lua for use in gameplay code
  • Implemented Lua wrappers for efficiently dealing with SDFs and OpenVDB (the library mentioned in the last log that we now use for extracting meshes from distance fields)
  • Implemented CSG functions on SDFs that are roughly 10x faster than OpenVDB's defaults, allowing complex, detailed meshes to be built very quickly (read: better procedural mesh algorithms coming soon!)
  • Wrote a simple pathtracer for fun since graphics Josh has been repressed :monkey: Not totally irrelevant to LT, though, as we could perhaps use it for nice renders of procedural assets or for having a 'ground truth' reference for our physically-based rendering code
  • Built several Lua 'template' applications that allow us to easily various, specific pieces of functionality really quickly (just like the old LTSL sandboxes!)
  • Loads of other things that I'm forgetting

Adam

  • Started working in the Lua side of the LT codebase!
  • Put in the time to learn about all of LuaJIT's lovely 'idiosyncrasies'; Adam is now 100% capable of helping me deal with the technical side of LJ
  • Used LuaJIT magic to implement the first parametric type in our CType system -- the arraylist! This means that not only can we define and use new, native datatypes in Lua, but we can also have what amounts to native std::vector<T> and so forth of them. I know I've beat this horse to the ground, but seriously, the power we now have to create and use types that are as fast as compiled code simply can't be overstated! Memory management is by far the weakest spot in Lua's performance, and we are almost entirely side-stepping it.
  • Hooked up BSP code to Lua and used it to implement picking (being able to click on an object in the world and do something in response -- a seemingly-trivial task that is not at all trivial for large scenes and complex meshes, since it requires casting a ray through acceleration structures)
  • Got our first star system set up and working using CTypes and all the new high-perf Lua stuff!
  • Made a lot of improvements to our geometry code (BSPs, triangles, rays, etc.)
  • Implemented really fast BSP-Sphere intersection tests
  • Loads of other things that I'm forgetting (again)

Sean

Sadly, last Friday was Sean's final day here as an intern, since he's now headed off to college to study CS and become an unstoppable force of programming. We'll miss him for sure...his time here seemed way too short. Nonetheless, he was able to accomplish an impressive amount in the seemingly-little time that he had.

In his final act of glory, Sean finished implementing a fully-generic, hierarchical, fleet-based AI maneuvering system, capable of representing fleets with any number of sub-fleets, and intelligently directing their motion. To demonstrate it, he showed us a group of several fleets flying together in a V formation, with each fleet comprised of squadrons in a V formation, each of which was comprised of fighters in a V formation...you get the idea! All of the math required to make it happen is rather difficult due to this arbitrary nesting requiring scale-invariance. Sean got it all hammered out and now we have proper, hierarchical fleets. I'm really excited to see this in-game :D I don't have a gif on-hand, but I'll be sure to show off his work as soon as we attach it to the main LT Lua code.

---

High-Performance, High-Convenience: A Peak at Coding in LT

As you all know, Adam and I have worked relentlessly over the past months to devise ways to achieve both extreme performance capable of handling the LT simulation logic, as well as the convenience of being able to write said logic in a language that won't rob us of every last drop of life. I'd like to show just a snippet of code demonstrating what our native type system ('CType' -- the final version of what we used to call the ECS) can do, how it looks, etc. So, have a slice of toy code! (Since it's harder to read without syntax highlighting, I just screenshotted some code in-editor)

Image

This is a massively-simplified battle simulator, intended only to showcase the simplicity of gameplay code. Nonetheless, it demonstrates some important features / fruits of our labor. Note: I cranked this out really quickly and didn't actually test it, there may be typos or slight mistakes :whoops: )

Notable Take-Aways:
  • Native memory types can be defined on-the-fly in Lua; behind the scenes they are 100% as compact as the equivalent type would be in statically-compiled C; accessing the memory is extremely fast due to fancy, custom memory-pool allocation magic done automatically by the engine. All of this complexity is hidden from the user, so code remains very easy to read and write.
  • Types can be nested in other types, enabling 'component-based' design, as demonstrated with Health and Weapon. Of course, since this is a full-blown type system rather than an ECS alone, we can do things like have multiple health components, which ends up being useful sometimes!
  • 'Member functions' can be attached to these types and called in the idiomatic Lua manner; convenience + performance = winning!
  • Generic functions like 'Weapon:damage' can be written without concern for the type of the target object and still 'just work,' despite the fact that we're using direct field access instead of table lookups like most dynamic languages (if you're a programmer and understand what I mean, that should blow your mind a bit :) )
  • (Here's the even-more-mind-blowing magic) Functions that can operate on different types of objects (like Weapon:damage, which could be used to damage stations, hardpoints, etc) will be traced by LuaJIT and end up running as fast as equivalent statically-compiled constructs (in C this would require a switch statement or function pointer, in C++ this would require a template or virtual tables; here it requires nothing extra!!); this is a HUGE win for both performance and simplicity!
  • Although I didn't show it above, types can be built in a piece-meal fashion rather than defined all at once. In particular, mods have access to type definitions and can inject their own fields and methods as required to support extra, mod-related data and functionality (I showed something like this in a devlog a while back). As always, giving modders the ability to add just about anything conceivable to vanilla LT is really important to me!

---

Well, once again I've completely and utterly failed to be brief :ghost: :ghost: It's hard to be brief when there's so much happening and so much about which to be enthused. Speaking of being enthused. I must leave you all now to return to the code :)

:wave:



Hello, everyone! Talvieno here with a non-technical summary. There hasn't been much demand for one for a while, but this devlog seems to be worthy of it. Without further ado, let's get down to business! There are a couple "big things" to take away from this update - namely, Sean is no longer with us, and we're almost ready to start focusing entirely on gameplay. As to Josh's bullet lists, I'll break down what those mean in the spoilers below.

Josh
(tl;dr: made things much faster and more performant)
Spoiler:      SHOW
  • Implemented stuff for better performance
  • Made it easier to write code while the game is running (I think)
  • Implemented more stuff for better performance
  • Hooked up pieces of code we've been working on for a while so we can actually use them, but then called it CType (instead of ECS) because that sounds cooler and makes more sense
  • Some stuff that printed stuff to screen that used to be in Lua is now in the C code core, so it's faster
  • Made major improvements to code that let us draw things that are far away (or very large)
  • Worked some on the foundation for gameplay code
  • Implemented stuff for better performance (Yes, again)
  • Implemented stuff for way better performance (No, really)
  • Got sidetracked and made some really shiny images with what seems to be a homebrewed renderer (which is particularly impressive, if not entirely LT-relevant, but it was done in Josh's free time)
  • Built some programs to let us test small pieces of LT Lua code quickly
  • Lots more where that came from

Adam
(tl;dr: Lots of coding wizardry)
Spoiler:      SHOW
  • Started working on gameplay code and gameplay-related things!
  • Fully educated himself on Lua and LuaJIT, making him roughly on-par with Josh's level of accumulated LuaJIT knowledge
  • Long complicated bulletpoint that essentially means Adam is a wizard and is now quite near god status. Basically he implemented a lot of techy workaround stuff that overcomes Lua's inherent flaws. Neato, right?
  • More wizardry. Like Josh said, Adam implemented stuff that lets you click "on" an in-world object. Trickier than it sounds. It's taken care of now. It seems this may be more robust than the way Josh did it before, too.
  • Created a star system with the new LT code he and Josh have been working on. I basically copied this from what Josh said, it's pretty clear already.
  • Implemented more stuff for better performance (yes, Adam is guilty of this generic task description too)
  • Implemented some really performant collision-related tests, meaning we can check for collisions even faster. (Collisions are actually really hard for computers to detect. Our brains are much better suited to it.)
  • Loads of other stuff that Josh is forgetting.
Okay, now we're to where Josh is telling us that Sean is leaving. :( We'll miss him dearly, and hope to see him around sometime in the future. In his final act of glory, he [technical gibberish that basically means he made ships in fleets fly around in an actually smart way, instead of all flying around haphazardly like in the LT Prototype]. It'll look pretty, believe me.


As to all the last part (the image, the code, the final bulletpoint)... that can be summarized by saying Josh is proudly showing off the fruits of his labor, which unfortunately may not mean much to anyone that isn't a well-established programmer that can at least code on an intermediate level. The main things to take away from it is that it's fast, and flexible, meaning it's the perfect tool for modders. I'm actually understating it here - this is some top-tier wizardry that Josh and Adam have written up. If you're used to games running considerably more slowly when you start adding mods, that shouldn't be the case with LT. For those interested in writing their own mods, all this also means that you'll have a ridiculous amount of control over the functionality you can add to the game.

And, that's it! Josh failed epically at being brief (does he ever not? We love him for it), but I've hopefully condensed it down to something a bit more pleasant to read for the non-technically inclined. :) I'll answer any questions as necessary, and thanks for reading!



Link to original: viewtopic.php?f=30&t=6271
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: The Limit Theory Email/RSS News Thread

#33
Thursday, August 31, 2017

Really happy to report that it's been a tremendously productive two weeks! In the last log I spoke extensively about our various mechanisms for getting bleeding-edge performance out of a magic blend of LuaJIT and C. Since then, we've been building more and more 'real stuff' using those systems, working our way to release-quality code! The general theme as of late has been "wow, it just works!" Of course, the longer version is "wow, after 1000s of hours invested in technobabble and aggressive banging-head-on-desk followed by burning the codebase and restarting, things finally, at long last, just work!" :ghost:

---

(I'm not going to separate by Josh & Adam this time since we were both working all over the place.)

Basic Game Objects & Components: Working!

Last week was the first time that we were actually able to build and use some real components and game objects purely out of our CTypes system (see last devlog if you don't know what that means). It worked beautifully. Right now we've only got System, Asteroid, Ship, and Nebula objects -- we're scaling up slowly as we nail down what we want the gameplay code to look like / how we want to be able to manipulate said objects in Lua. We've also got some basic components -- Motion and Transform, in particular. The former handles kinematics, the latter keeps track of position/orientation, etc. These are your bread-and-butter components for a component-based game engine (along with something like Drawable/Renderable, Inventory, etc.), so naturally we want to test the most ubiquitous components first.

The real stress test of our CTypes system came when we packed thousands of asteroids into a system, doing a rather naive update loop on them, and performing full rigid body dynamics on them (meaning I stuck a motion component in the asteroid definition and allowed every single asteroid to have accurate linear and angular dynamics -- something that we're probably not even going to do for release, because I still contend that dynamic asteroids don't add a whole lot of value..but hey, mod away). The speed was unreal. The bottleneck was, by a huge margin, the rendering (no LOD models yet). The actual update logic is so fast that, if I turn off rendering, I can get up to about 30,000 asteroids before I drop below 60 FPS on my machine. Again, that's a full rigid body dynamics step happening on 30,000 moving, spinning asteroids @ 60 FPS. And that is using very naive update logic, no cleverness about putting objects that aren't moving to sleep, no threading, etc. Adding in cleverness is only going to push that number even higher :shock: Anyway, I guess I can stop bragging about our performance now :oops: It's just that...I spent SO long battling FPLT -- and sometimes seriously doubting that I'd be able to give LT that 'epic' scale that I so badly wanted -- that...surely I deserve a little leeway... :monkey: :angel:

Dynamics: Fast, Accurate

I mentioned the rigid body sim above, but it's worth mentioning it as a stand-alone point. To be perfectly honest, I've never had accurate physics code in LT. I mean, the linear part was accurate. Kind of. But angular dynamics have always been extremely hacky in LT. But those days are over, and we now have a genuine physics integrator with accurate angular dynamics (i.e. applying impulses at various points on objects results in correct torque & force). Honestly it's not really a big point, as you likely won't notice many cases of fancy angular dynamics in LT. But I think the takeaway is that this stuff was written in Lua (yes, the physics step is in Lua...!) and runs well within the acceptable range for us to support a massive simulation.

I suppose it's also worth mentioning that mods could certainly leverage this to 'go further' with the physics in LT. You all know I'm a fan of realism right up until it starts cutting into fun, which, in my experience, happens quite early-on. I don't intend for ship thrusters to use real impulses, because I don't want players to have to spend hours trying to properly mass-balance their ships to get inertial characteristics that aren't impossible to fly with. But for those of you who love that good old abused phrase "real Newtonian physics" -- well, mod away. I already wrote the real physics for you ;)

Adam's BSPs are Stupidly-Fast

Several times over the past week I've had to turn around and tell Adam that his BSPs are just grossly over-performant. In our current LT sandbox, we can click objects to select them, view data about them, orbit them with the camera, etc. Selection requires raycasting the scene, which, in turn, requires raycasts against the BSP trees that provide the acceleration structures for meshes. Currently I am not using a broadphase structure, so we are literally raycasting against every single asteroid / ship in the scene. Really, it shouldn't even work. It should just melt the computer when you have thousands of complex objects in the scene. But it doesn't. In fact, the fps hit is so low on my machine that I forget how negligent I'm being.

The biggest place that raycasts are used (biggest in terms of 'most likely to cause performance problems') is in checking to see if a projectile weapon hits an object. Pulses, beams, railguns all use raycasts. Missiles are slightly more complex but still do some raycasting.

I guess where I'm going with this is: we can probably support like 10000000 million projectiles in the world at once. Or something like that. Basically, REALLY BIG BATTLES. 100 ships is going to be a cakewalk, apparently :shock:

Josh's Nebulae are Stupidly-Fast (and Pretty!)

I spent some time after-hours one night revisiting my most-prized algorithm: the nebula generator. I made it roughly 10x faster while also improving the visual fidelity AND making the generator more consistent, such that it yields less "really bad" systems. It was a fun night. We all know I'd die if I went more than half a year without touching 'nebula.glsl' :roll: But seriously, I'm happy about this, because nebula generation was one of the most expensive parts of system generation (by quite a lot). It's now much more reasonable, especially on less-powerful GPUs.

Render-Time Interpolation of Simulation State

We implemented it. This is a major technical component of games that need to have tight control over the game world simulation (like those that want to simulate universes). I've spoken about this in devlogs long, long ago. The idea is that simulation of the game world happens at a fixed rate, independent of the frame rate. This makes simulation more stable and typically more performant as well. It also brings a major challenge: your simulation state is no longer in-sync with your render loop, so now you have to 'guess' what the simulation state would look like at the current render time. To do so, you simply keep one frame's worth of historical state, and interpolate between the last and current state to obtain a smooth, linear approximation that gives the player the perception that everything is nice and continuous on-screen.

Major game component, done.

Dev UI

We've built (well, mostly Adam) a really handy developer UI to help with poking and prodding the game as we work on various parts of it. We decided that the real UI should be one of the last things we do (this was guided by the realization of how much time I've sunk into UI that later became useless when underlying game mechanics changed). Until then, though, we've got some nice programmer-artsy widgets to see what's going on.

Multiplayer: Just for Fun

Last Friday I implemented multiplayer. Just for kicks :ghost: Don't get excited (or worried), it's not a feature and we're not shipping with it. By multiplayer, I mean I implemented a very naive peer-to-peer network architecture so that Adam & Taylor (a good friend/fellow programmer at the tech park) & I could all throw asteroids around in the same star system. We would find the biggest asteroid that we could in the system, then set our cameras to orbit it, then fight to push/spin it in various directions with impulses. Not exactly MMO-of-the-year material, but it works.

So, the UDP socket API in our engine works, and I've proven that it's possible to implement (some degree of) multiplayer with it. I hope the LTMP modding team is ready! :V (Also, I was on linux and they were on windows, so it works across different platforms, as you would expect of a decent implementation.)

---

Conclusion:

Everything is coming together. The road has been thoroughly and meticulously paved, after oh-so-many setbacks, and we're FINALLY getting to spend almost all of our time in Lua at this point, just pushing forward...which is a really good sign! Game constructs are being built and carefully probed for performance along the way, ensuring that everything is done 'the right way' so that (ideally) we don't end up with any nasty performance-related surprises in the end. That which is being built is being built to last. Thus far, all of that effort we sunk into performance is REALLY paying off, because the work is moving super quickly.

I know it's probably difficult to appreciate how much progress the above points represent -- especially when screenshots don't look appreciably different than they did several months ago in the early days of LT in LuaJIT. But if you've been reading, you know the truth: under the hood, it's a whole different ballgame. This time, the muscle under the hood is enough to get us to release. I'm genuinely excited about it (and let me tell you, it's not easy to excite someone who's spent the last five years doing little more than staring at vim :lol: )

From here, we just keep at it. Every day, more game. Every day, more LT. Then one day...

...you know :)

PS ~ Some eye candy just because. Nothing you haven't seen before -- pretty nebulae, decent asteroids, bad ship algorithms...probably too much bloom :oops: But I know you guys require food every now and then :)

Limit Theory Devlog Gallery ~ August 31, 2017

Image

Image

Image

(More in the gallery)



Link to original: viewtopic.php?f=30&t=6281

((Talvieno's note: There is going to be a Kickstarter update very soon. I was initially going to wait for it and put them both in the same post, but I've opted to post this now, and then I'll be making another post with a copy of the Kickstarter post when it arrives.))
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: The Limit Theory Email/RSS News Thread

#34
Monday, October 2nd, 2017

Ahoy there! I know it's been a long time since Josh made a post, so I'm stepping in here for a moment with one from Adam, just to show progress is still being made. For those who don't know, Adam is one of the programmers Josh hired, and this is his first major post. We'll get back to you soon with more regular updates from Josh, but in the meantime, hopefully you can enjoy this one. :)
AdamByrd wrote:
Mon Oct 02, 2017 11:31 am
Hey guys. I'm going to try something new for a bit and see how it goes. In between Josh's bigger updates I'm going to start posting smaller updates. These mini-updates will focus on the specific problem I'm solving at the moment. The goal is to go deeper into detail than the bigger updates normally do and is often going to be centered around code architecture since I've been spending a lot of time there lately. They will also be more informal: I'm going to write them without much revision and without the flair that generally comes with Josh's writing. This is mostly just laziness that I'm selling as time savings and pragmatism. I also imagine this being a little more interactive than the usual updates, so feel free to ask specific questions or tell me what you'd like to hear more about and I'll do my best. Off topic stuff is fine as well as I realize you guys don't know a whole lot about me yet. What I won't be covering is what Josh is currently working on. For one I don't want to steal his thunder and two I want to ensure there's still new information in the regular updates. Onward!

Recap
First a bit of history. My primary goal since teaming up with Josh has been to get to gameplay as soon as possible. In a particularly productive team meeting early on this lead to the birth of Lil'T. The original goal was to have a single, playable system in a week. Something basic with player controls, a nebula, and some asteroids. This was back when I had just finished the BSP implementation. We had been working solely in the engine and didn't even have a separate (code) project for LT. I didn't know any LUA or how to start writing gameplay that leveraged the engine so this was a good motivator to start knocking down those barriers. I think it ended up being about 2 weeks before we really got to the point we wanted to be. I have to say, hooking up the BSPs for the first time and being able to click things in the scene felt really damn good.

Now, as much as I want to get to gameplay, the second major goal is making sure the code we write is shippable. It can't be prototype stuff that's going to need to be re-written. It doesn't have to 100% perfect and complete either, but it needs to be written in a way that can expanded without needing to be refactored. In getting the first phase of Lil'T running it became obvious that the infrastructure to support LUA needed to be in before going further. That took another few weeks and you've likely read about some of that in the past couple of updates. During that work we spent a lot of time profiling, testing, and tweaking things and it became pretty obvious that being able to do some of that while the game was running would save us a bunch of time so I set to task on the debug UI.

With the dev UI in a good spot we're shifting back to gameplay again. We're likely going to do this 1-2 combo of sprinting on gameplay for a bit then stepping back to improve the infrastructure supporting it a few more times. Often we can't really tell what the scaffolding needs to look like until we have concrete gameplay as a guide. We spent a couple of days talking through how we wanted to simulate entities. Since they have one-to-many parent-child relationships (a ship is the parent of it's turrets), the game objects form a tree structure. We compared hierarchical updates using depth first and breadth first traversal. We compared job-based (run on all entities) and event-based (build a list of entities that need to be updated). Ultimately we settled on event-based updates with the ability to create jobs within that framework.

Input vs Simulation
Once we nailed down how we wanted updates to be processed, input became the next target. We ended up with a lot of code that looks like this:

Code: Select all

function onUpdate (dt, hasFocus)
  if hasFocus then
    if PHX.Mouse.Pressed(PHX.MouseButton.Left) then
      self:fire()
    end
    if PHX.Mouse.Pressed(PHX.MouseButton.Right) then
      self.camera:setTarget(self:getPick())
    end
  end
end
(Note that I'm being loose with the term 'update' in the text (but not the code). The core game loop is essentially simulate, update, draw. Simulate is the fixed interval simulation step. Update happens right before draw and is for things like animation and interpolation. Draw is for, well, drawing. Also note that handling input in the update step is wrong and will be changed eventually. We aren't currently buffering input and since the simulation rate is much lower than the update rate we miss a lot of input when handling it in update at the moment.)

This isn't great. Having to manually track, pass, and handle focus in every single location that processes input is going to lead to a lot of mistakes and unnecessarily obtuse code flow. We knew this wasn't going to last long as it was being written and it needed to be addressed soon, before enough incorrect code built up that refactoring it would be a big task.

Josh and I talked through this quite a bit and his idea was to treat the game window as a fairly standard UI. The GameView would just be a UI widget that displays the final, rendered view of the simulation. The simulation becomes an autonomous thing that marches through time, updating its own state without any concept of a camera, a player, or input. UI widgets will provide visible controls and input handling that inform the simulation of requested state changes. The GameView decides how it should render the view of the simulation.

It took me a bit of time to understand the implications of this approach. It sounded a bit odd at first, but the more I worked through it, the more I was able to see the large number of subtle benefits.
  • The GameView gets the full benefit of the automatic layout functionality of the UI. Debug and gameplay UI elements can push the GameView out of the way in addition to just overlapping it. If you've noticed how the debug UI overlaps gameplay that's actually quite annoying. It's nicer if the debug window is able to make the gameplay view narrower and slide it to the right so they are side by side instead. In general it means we can allow the game view to be resized, moved, snapped, etc very easily and naturally. We can iterate on UI layout faster, allow players to customize their UI easily, and handle window resizes without any added work.
  • Multiple GameViews can exist (think radar, scanning, minimap, whole-system views, etc).
  • Input focus is already handled cleanly. The UI tracks what your mouse is over, which widget has focus, is being dragged, etc. It's trivial to extend this to have separate mouse and keyboard focus, hard and soft focus, separate onInput functions for letting only the focused widget handle input, and whatever else might be needed.
  • The amount of branching in gameplay code drops significantly. We don't have to check for focus or input because that gets hoisted to the highest level and doesn't need to be repeated all over the place. Even with our tiny system, the amount of code and complexity immediately dropped.
  • The simulation takes on more of a client-server model. Input is handled in the 'client' which then requests state changes on the 'server'. Validation in the simulation all happens in one place. We end up being forced to build a clean API in some sense.
  • Concepts like control schemes become much easier to think about and implement. Since each form of input is going to be a separate UI widget completely changing how the game controls just means creating a new widget and adding it to the UI hierarchy.
There's probably a lot more nuance that I'm leaving out. Anyway, we're currently using the dev UI to handle this. We didn't originally intend for the dev UI to be the final UI system but the performance is good enough that it's actually a possibility. It's powerful and easy to use. However, if the dev UI is swapped out in the future it's not going to affect the way the gameplay code is written so it's natural to move forward with it for now.

Of course, making this change meant a decent amount of refactoring. I had to pull all of our input handling out, decide where it went and reimplement it in the new system. I ended up with 3 new widgets: GameView, DebugWindow, and DebugPicker. I also reimplemented the data inspectors Josh added a while back so we can actually walk through the entire game state from UI windows. I also took the time to add a bit more UI polish: added stretch weights to grid columns/rows, fixed subtle positioning errors in labels, added alignment to labels.

This is mostly under-the-hood work, but I figure I can at least drop a gif of the GameView being animated smoothly: https://i.imgur.com/EkASTsN.mp4


This update is a bit higher level than I'm aiming for in general, but it's not a very involved or probably interesting topic and it's only a few days worth of work. Let me know what you guys want to hear more/less about so I know where to focus in the future posts.

Cheers
Link to original: viewtopic.php?f=30&t=6310
Post

Re: The Limit Theory Email/RSS News Thread

#35
Wednesday, October 4, 2017

Hey again everyone! 'Tis once again the season for Josh to apologize for waiting too long to post a new devlog... :ghost: Honestly, time just flies when you're having fun working on LT. And we've been doing a lot of that.

Talv, in his usual beyond-his-years Talvish wisdom, has advised me that it's more important to be consistent with devlogs than to be exciting, flashy, etc. I feel like I've heard that from you guys before. At some point, it may actually sink in to this thick head of mine :oops: At any rate, I'm going to turn over the 1-hour hourglass (that's not redundant because I also have a 30-minute 'hour'glass...) on my desk and force myself to finish this log before the last grain hits the bottom.

To accomodate the expedited writing process, I'm going to bullet-point the list of what I've been working on, then expand on a single element of focus (whichever I feel is the most important at the moment). We'll see how it goes!



Since Adam has already been kind enough to give us quite some detail on what he's been up to, I'm just going to go over my own work. So...over the past month, I've:

  • Implemented several new components for entities, most of which are in preparation for / in service to implementing gameplay logic
  • Implemented a mod-loading system that performs validation and loading of isolated mod files, which of course means that I also...
  • Defined a mod file format and started writing logic in mod form; ideally, most if not all of our actual gameplay logic will be written like this, both for our own convenience in being able to easily plug/unplug pieces of gameplay, as well as to ensure uniformity with respect to 'built-in' content vs. mod-added content
  • Implemented an event-handler architecture for effortlessly broadcasting events as well as hooking listeners to respond to said events
  • Worked with Adam to implement first pass of 'job' system, which is, essentially, the 'code' analogue of an ECS
  • Implemented several jobs / pulled much of the existing LT logic out into jobs, resulting in isolated, efficient, and easy-to-reason-about logic
  • Implemented (naive) collision detection and resolution (as a job) -- had lots of fun with large quantities of asteroids bashing into one another :D
  • Tackled many, many performance optimizations accross the board, attempting to ensure that we stay lean-and-mean as we push more content into the game; some examples are LOD meshes & automatic generation thereof, fast font glyph caching, fast vertex AO computation for SDFs, and more...
  • Explored building a simple editor program for expediting the process of adding components, entities, and logic + rolling those changes into a mod file and testing it in-game (heavily-inspired by the TES games and the legendary 'TES Construction Set' from back in my Morrowind modding days)
  • Began work on version 2 of the jobs system, which will, hopefully, unify how we write the heavy, running-almost-all-the-time gameplay logic and the sparse, happens-only-under-specific-conditions gameplay logic
  • Got the 64-bit Windows version of LT built & running



Jobs and Gameplay Logic

With our CTypes system firmly nailed-down and providing an excellent solution to how we handle data in the game, we're now looking to solve the equivalent problem with logic/functionality/code. After all, there is only data and function, nothing else :) With entities, we asked the question: how can we efficiently, easily, and extensibly define the format and manage the lifetime of the data that keeps track of an entity? How can we do so in a way that will support many diverse types of entities, while also supporting the potential to have hundreds-of-thousands of these entities in play, and perhaps even thousands or tens-of-thousands going into and out of existence in a matter of fractions of a second?

To those questions, we answered and answered with vigor in the form of an ECS, which became an ECS v2, which became CTypes.

Now we ask the analogous questions with respect to logic: how can we efficiently, easily, and extensibly define the format and manage the lifetime of the logic that runs on and creates the characteristic behavior of an entity? How can we do so in a way that will support diverse types of entity behaviors, and potentially hundreds-of-thousands of such behaviors in play at once, and potentially tens-of-thousands of these behaviors stopping, starting, or being replaced with a different behavior at any given moment?

I've touched on partial answers to this question in previous logs. Part of the answer is: exploit the sparsity of game logic to do 'as little work as possible' at every moment of time (note the symmetry with how we store 'as little data as possible' in CTypes by using native memory, pools, static typing, etc.) This means, for example, that an AI player who is travelling to a location does not enter a state where the script constantly checks whether the player has arrived at the destination. This is wasteful and fails to exploit the sparsity of the situation (the probability of arrival is exceptionally low on any given frame). Instead, the script informs the engine of the specific condition(s) upon which it should be resumed: "Dear engine, I am writing to inform you that I no longer require CPU cycles until my client, Sir NotEdisonTrent III, aka 'Trent', arrives within 100 metres of the asteroid @ memory address 0xA513701D. Alternatively, if my client should be the victim of an attack, I would like to be notified in spite of not having arrived at said asteroid. Thank you for your time, see you in a few billion cycles. Love, script/ai/TravelTo.lua." If the underlying logic engine is clever with respect to how it services such a request, then the difference here is the difference between being able to run a few hundred AI units vs. being able to run a few hundred thousand (not that we need to run a few hundred thousand -- but keep in mind that if we consider all the other logic going on in the game, then indeed, we are looking at on the order of a few hundred thousand bits of logic). It's an orders-of-magnitude subject, like many things in computing.

We already have a job system for easily running logic on large batches of entities that is very cache-friendly as well as easily-parallelizable. At the moment, I am working to fill the final piece of this puzzle: enabling said system to accommodate highly-sparse logic like the example above. The good news is that I have already designed (on paper) the architecture for the system. I'm working on the implementation as we speak (well, I mean, by the time you read this I'll be back to working on it)! Interestingly, implementing such a system is, in many ways, symmetric to designing a robust OS kernel, as many of the concepts have parallels in OS design: a piece of logic is a thread, sparse logic is a thread that waits on condition variables (a fundamental synchronization primitive used in low-level kernel / driver implementations), thread groups are woken by condition variable signals / broadcasts, a job is a process, each instance of a job is a thread, job state is thread-local storage (TLS), ...and so-on (of course it's much easier to do for a game than for an OS, don't be alarmed by my comparison!) Lucky for me, I did manage to pass the operating systems course at uni before dropping out ;) (Side-note: our professor was the guy who made one of the first scripting languages, Tcl. He was the best prof I ever had.)

An infrastructure for handling game logic is the last remaining element that I see between where we are now and a time where we can iterate wildly on shippable gameplay code. It's all very exciting :D



Success! Would you look at that? I finished just in the nick of time (ok, maybe by the time I proof-read and actually post, I will have gone a few grains over :roll:). I realize that you all probably would like more details on some of the other elements, but I'm clearly unable to give 'brief' summaries of things when I'm excited about them. And I'm excited about a lot of those bullet points. At least this way I'm able to write logs in a reasonable amount of time while still providing juicy details + a high-level picture of progress.

I really would have liked to explain a bit more about my work with the proof-of-concept editor, but alas, it would require another turn of the hourglass. For now, I just want to mention that it is being 'mostly shelved,' as the goal was to determine whether the cost-of-implementation of such a tool would be less than or greater than the time saved developing. From my exploration, it seems the cost-of-implementation would not justify the LT-dev time savings, so we will most likely not see continued work on the editor. I can imagine some pragmatists out there cheering and some dreamers out there crying :roll: :ghost: Well, come on, we have to save some things for LT 2, right? :lol: That being said, it still has some useful features: we can use it to easily get output from LuaJIT tracing & assembly dumping on Windows, where it's more difficult to do so than on Linux.

Now then. I'm going to hold myself to posting the next devlog next Friday. Period. Next Friday I will do the same thing that I did today: turn the hourglass, and write. Talvieno is right, as always: consistency is the key. And if I just wrote a devlog more often, I wouldn't have to squish so darn much into one anyway!

Oh dear. It appears next friday is the 13th. Ha. Oh well, numbers are just numbers, and, after all, 13 is both a prime AND a member of a twin prime pair, so I don't see how it could possibly be unlucky. See you all on the 13th... :ghost: :wave:



I took a few screenshots to accompany the wall of text: Limit Theory Devlog Gallery ~ October 4, 2017

Wireframe view of an asteroid field, where you can see the LOD mesh optimizations (notice the wireframe density remains approximately-constant in screen-space, despite asteroid size or distance. Constant screens-space detail is 'ideal' LOD.)

Image

I wrote a simple mod to shoot asteroids out of my ship. I used it to test the new collision detection & resolution job. It's fun blowing apart a big sphere of asteroids with a spray of little ones :nerd: (It's kind of hard to see without an animated GIF, but I'm not nearly as good as Adam at making those..)

Image

Using Plume (the concept LT editor) to get LuaJIT assembly dumps on Windows:

Image




Link to original: viewtopic.php?f=30&t=6315#p155467
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: The Limit Theory Email/RSS News Thread

#36
Talvieno's note: I accidentally missed a devlog, so I'm going to put that one, and the most recent one, into the same post. Please be aware they're two separate posts, spaced about a week apart. I apologize for this and will do my best to ensure it doesn't happen in the future.




Friday, October 13, 2017

In the last log I promised that today, I would turn over the hourglass and write a devlog. So that's what I'm going to do :geek: Better get going, those grains are relentless.



This week, I...

  • Implemented a spatial hash grid structure similar but far superior in performance to what LT C++ used to use for physics
  • Added xxHash to the engine; replaced several internal hash function usages with xxHash to reap the performance benefits; did some performance benchmarking to confirm how awesome it is (thanks to Adam for bringing its existence to my attention)
  • Used the aforementioned spatial hgrid to implement broadphase collision detection; despite being in Lua and still being quite non-clever (other than hgrid usage), it performs remarkably well
  • Implemented a generic C-style hash map with open addressing and lossy hashing for ultra-fast lookups (profiled @ roughly 5x the GCC 4.8 libstdc++ unordered_map impl; not that it matters); will be used in the near future for gameplay-related code
  • Unified parent/child relationships in CType components
  • Introduced and experimented with a new pattern of 'components as functions' -- since the CTypes system has 'type descriptors,' we can add fields and methods via normal functions that operate on type descriptors (a very dynamic-language-esque thing to do); doing so may simplify component logic -- we'll see!
  • Implemented powerful new mechanisms in the CTypes core to support automatic constructors, destructors, initializers (with arbitrary arguments), and automatic reference-counting
  • Used said mechanisms to simplify a lot of code
  • Used said mechanisms to help implement the Item and Blueprint classes in CTypes (I'm itching to get rolling on gameplay, so I've started working on adding some of the gameplay-related datatypes and such!)
  • Implemented two (very) different types of script manager to explore options for writing gameplay logic
  • Wrote a simple AI test script in each script format to explore usability, performance, concision
  • Took some time, while I was at it, to implement a new AI maneuvering algorithm that uses the PID (proportional-integral-derivative) controller algorithm to robustly achieve a desired position and orientation with minimal stupidity; results are really encouraging...the AI is better than ever before at piloting! (That is, in terms of the basic stuff like adjusting course and thrust. We still need to pull over Sean's work to get real formations and collision avoidance and so forth!)

Last week, I talked a lot about building a system to efficiently support gameplay logic. I could continue that subject and talk at length about how I went about doing so this week...but I want to talk about new things. Plus, I'm sure that the scripting engine is something you'll hear about frequently in the coming logs, so let's not beat a dead horse before it even dies ._.

Instead, I'll just talk about the AI logic that I wrote today using one of the two script architectures that I built this week.



AI Maneuvering Using Proportional-Integral-Derivative (PID) Control

A friend at work who is doing some real-life AI piloting coding (for a quadcopter drone, nonetheless) mentioned this 'PID' algorithm to me. I had never heard of it before, but it sounded interesting. Like the vast majority of my memory, it slipped away below the visible horizon of my consciousness until today, when, while trying to write some representative gameplay code to test out the script scheduler, I was messing with AI piloting (read: watching really dumb AI ships thrust themselves into oblivion, or, if lucky, get caught in a headache-inducing micro-orbit, doing donuts in space at 10Hz). That's when it surfaced, by chance, and I went to ask him more about AI piloting algorithms. A few hours and a bit of math later, AI ships were forming up on me in record time, with an all-time-low of overshoots and micro-oscillations. Even the small, drone-sized ships that currently have an absurd angular-thrust-to-inertia ratio (yes, these were the ones doing donuts...) and can't even be piloted by yours truly are stable and well-behaved. The AI now rocks at piloting mechanics. The magic was all in the PID controller algorithm.

PID is a really elegant way to approach 'black box' control. By that, I mean that one does not have to know all details of what is being controlled. In the case of AI, we do, of course, know what actually happens when a thruster is fired: an impulse is applied to the ship at a specific point, resulting in a net torque and net force on the ship, resulting in a change in linear and angular velocity, resulting in the ship changing position or orientation or both over time, etc. We know that there is some drag, both linear and angular (because Josh doesn't think space jousting is fun; if you do then comment out the two lines of code that add drag to the game instead of yelling at me on twitter :ghost:), we know that ships have some mass and maybe an anisotropic inertial tensor that will affect what happens when we thrust.

Yeah, we have a lot of knowledge. But to take all of this garbage and make it into an algorithm that gets a simple ole' mining ship from A to B? Surely it isn't so hard. Surely we don't need ten chalkboards-worth of archaic greek symbols just to make sure that Mr. Node the Nodely of Nodooine can point his ship at a rock and mine some Dense Nodespar without igniting three of his four engines before slamming into a rock and dying a violent space death due to some 'minor' thrust miscalculations.

I mean, what do you -- presumably a human (if not, hi Taiya, glad you finally achieved sentience \o/) -- do when you go to control something in a new game? Figure out what physics equations the game uses, then use them to solve for the button/duration-of-press required to attain a certain position/orientation/steam achievement? Of course not! You're both much less clever and yet much more clever than that at the same time! You bang your hands and face against the keyboard until that little box pops up with "Achievement Unlocked: You have walked 5cm!" (Obviously a joke, no game would congratulate you for walking a certain distance... :ghost: :ghost:) How did you pull this off?? Well, maybe you figured out that if you press this button, you go forward. Maybe you pressed it too hard or for too long and now you're going too fast or have overshot your target, so you back off a bit, and repeat this process with the rest of the controls until you're 720-noscoping noobs through walls with an AWP while moonwalking and bunny-hopping at the same time. Ah, learning. :ugeek:

PID attempts to replicate the coremost features of that control process: identify a desired change (a proportional error), vary a control output with that desire, keep track of how said variation affects the rate of change (derivative) in error, and keep track of total error accumulated (integral). Using these factors alone in a simple equation is sufficient to achieve high-quality control over many processes. If you add a second derivative term, i.e. keep track of the rate of change of the error derivative, you can achieve even finer control over (non-linear) processes. This is exactly what I used to implement the new AI maneuvering algorithm, and it works beautifully -- all without having to touch specific knowledge like thruster output or drag constants.

I'm interested to see if this algorithm can be applied to higher-level bits of the AI, like project management. I'm also interested to get Sean's AI code pulled over so that we can have obstacle avoidance and formations. The good news is that, from these tests, it looks like this gameplay logic solution is viable. That'd be fantastic, because it's turning out to be super-easy to write logic with the candidate systems I built this week. More performance testing is necessary, though, since I only just got some real logic going on today.



Yayyy for sustainable devlog writing! Let's keep at it!

See you next week :)

PS ~ I know I hardly mentioned Adam at all in this log, but he's hard at work as well. I figure, since he's now officially posting devlogs of his own volition, and I'm trying to keep my logs more frequent and less overwhelming, I'll leave his work for his own logs. I will say, however, that he has done some really good stuff this week both with working toward various pieces of gameplay (ship hardpoints, turret control, WIP mining) as well as hammering at low-level things (we have a memory usage graph now! And it's already proven its worth by showing us where to optimize things that needed it badly :ghost:)



Yet another screenshot gallery is in order for this devlog! Just some shots of physics and AI from this week.

Shooting new asteroid fields into existence with the asteroid gun and broadphase collision detection (5K+ asteroids all colliding with one another at >60FPS!)

Image

A thousand AI friendsss using PID to follow me in formation with really nice piloting. I should really cap an animated gif of the AI in action so you guys can see the piloting in motion. Will try to get one soon.

Image

Ready to dominate the universe :X

Image




Link to original: viewtopic.php?f=30&t=6326



Monday, October 23, 2017
Hey guys! Since I've got a bit more than usual to say today, I'm just going to hop right in :squirrel:



Brief Recap of the Week:

  • Added physics 'sleeping' for dynamic entities -- dynamic entities at rest now take the same CPU overhead as non-movable entities; huge performance gains in a highly-dynamic scene with isolated pockets of kinematic activity
  • Added attachable component for external modules like turrets, thrusters, etc.
  • Implemented batched attachable updates for efficiently maintaining constraints between attachables and their parent
  • Moved existing placeholder turret & thruster logic into the real framework of our batched updates
  • Major improvements to the Job system (which handles the batched, per-frame logic)
  • (Experimental...) Removed frame interpolation; switched (back) to variable-dt updates in favor of cleaner/simpler architecture
  • Major cleanup of lots of game code due to the above!
  • Ported alpha pass rendering; ported beam shader (working turrets, hurrah!)
  • Extensive continued perf-testing of game logic overhead with an emphasis on scalability in high-entity + high-logic workload situations
  • Implemented and used 'simulated lag' and 'simulated stutter' to test performance scaling for low-end systems (the simulated stutter is a little too good...it'll make you queasy if you crank it up all the way :ghost: )



Sleepy Physics Awakens New Gameplay Opportunities!

Physics sleeping is a major and notable improvement on all of the previous incarnations of physics in LT, and guarantees that we'll be able to handle way more motion than before. I'll spare all the technical details concerning implementation, but the concept is very simple: don't spend time doing physics on things that are at rest (not moving). A decent implementation of this can actually eliminate essentially all cost of movable objects that aren't moving, meaning you can have lots more movable objects and/or lots more existing types of objects be movable. I say essentially only because movability requires extra information, extra information requires extra per-object memory, extra per-object memory induces extra cache misses when performing operations on those entities, so there's still a tiny up-front cost for being movable. But it's tiny.

One of the great results of this feature is that (subject to performance impact remaining minimal)...asteroids can be dynamic objects! I remember back in the early days when quite a few people were disappointed that asteroids would be unmovable. It seemed like a small issue to me at the time, but now that I've actually got them...it's quite nice :D There are several major gameplay wins at stake, including the 'big ship meets little rock' scenario (something that has caused me and, I imagine, a good many others here quite some pain and financial loss in other games... :ghost:) Previously, a large ship moving through an asteroid field was not just dangerous; it was a nearly-suicidal proposition. A carrier-sized ship hitting a static object 1000x smaller than it is an inherently-problematic scenario. Since an unmovable object has effectively-infinite mass, it can apply an effectively-infinite impulse to colliding objects. If one calculates a proper collision impulse and the resultant damage based on momenta and relative mass (as one should), then all of the kinetic energy of a hulking ship creeping through space can end up being reflected right back on it as kinetic damage at the hands of the tiniest fleck of unmovable space trash! Tragedy ensues as the fate of a faction goes up in flames, all thanks to some careless pirate tossing an empty beer can out the cockpit :ghost:

One can, as usual, come up with various ways to work around the issue, the most immediate of which would seem to be: destroy the debris. Not bad. But it's still unclear what happens when a ship meets something, say, 10x smaller. Or 5x. Or 2x. At what threshold of relative mass do we make the collidee disappear? Isn't it going to be weird having huge rocks vanish in a cloud of dust? Can AI players troll one another by building massive-yet-cheap hollow spheres of metal and launching them in the direction of a competing faction's primary source of raw materials, soon thereafter causing the whole field to disappear in a few glorious moments of poorly-resolved collisions? It actually sounds rather cool :ghost: Still, the resolution of the same situation with dynamic objects in play is even cooler: now the massive trollololosphere would drive a cylindrical hole through the field, dispersing asteroids outwardly and infuriating that one old miner who's been cutting power to his engines to get a tad more juice to the transfer beams. Returning to the more realistic game scenario, a battleship pilot need no longer worry about voyaging through an asteroid/ice/debris/cardamine field. So long as they avoid the largest obstructions, the impulses from clearing smaller obstacles will be easily handled by the shields (but please, have some respect for the poor folks who had to paint your capship -- do pack a shield generator).

It's still unclear if this will also enable mobility of large stations (the larger the scale, the larger the physics performance impact during motion; some back-of-the-prefrontal-cortex calculations tell me it's roughly-proportional to the surface area of the object, so we're dealing with superlinear slowdowns here). Frankly I don't see a whole lot of gameplay value in it. As far as I'm concerned, if you're a capital pilot and you're zipping toward Big Jiub's Quantum Armament Emporium at warp 9, you deserve whatever fate the physics engine deals out :roll: Since the issue is purely one of scale, having weapon platforms or even small outposts be movable -- and at the mercy of behemoth-class vessels -- would seem to be doable.

I've already done some really fun stress-tests with the system by spawning 1000 AI followers (running the PID algorithm discussed last time) and zipping through large (~10K) fields (at 60+ FPS!) Though the pilots aren't yet using Sean's avoidance algorithms, it's actually really neat to watch the formation as a whole either bump into (and subsequently compensate for) large obstacles, or clear smaller ones out of the way. Having everything fully-movable and collidable at the same time turns what would otherwise be a boring scenario into an entertaining one! Compare this with LTC++, where I had to (in the later days) turn off collision on NPC ships and force immobility of asteroids to maintain a reasonable framerate :( Suffice it to say, I remain very excited at all of the gameplay doors that are opening up thanks to so many hours spent fretting over CPU cycles :monkey: :D



Lindsey Joins the Dev Team!
Promises "At Least 300% Reduction" in Ship Boxiness!

That's right, there's more! I know you all are already over-the-top with joy after reading the above, since, after all, physics was the #1 most-anticipated feature of Limit Theory :roll: :ghost: But if you'll allow me just a few more paragraphs, I've yet another log to throw on the hype fire!

The third desk here at the office has remained unsettlingly-empty since Sean's departure. We can never hope to replace Sean's zest for flow-field-AI mechanics, nor would we dare try. But I'm very pleased and excited to announce that our third desk has found a new occupant: Lindsey Reid, technical artist extraordinaire.

Lindsey comes from an interesting mix of background experiences. Like Adam, she's worked here at the tech park in the past and been part of the game development community here in Baton Rouge, which is how we met in the first place. Some of you have already met her on IRC or the forums -- we've been dating for quite a long time now :) She's had the experience of working as a game programmer at a major AAA studio (making her...yes...the only one of us three to have worked in 'the big leagues'). When she approached me with a desire to work on Limit Theory, it was her knowledge of the juncture of art and code that sold me on it.

Lindsey's foremost area of interest is in improving the procedural generation algorithms, particularly for stations, ships, and artificial entities in general. Rejoice, for the time has come to retire Josh's "Ode to Box" generators and make way for a new generation of algorithms! As someone with genuine artistic talent alongside an ability to code, Lindsey can do something that I can't: she can build space stations and spaceships on paper, generating them manually with sketchpad-and-pen while simultaneously analyzing her own creative process and translating it into code. With assets in which both functional design and aesthetic form are key, this approach is more effective than my own process of rapid guess-and-check. Iterating on equations works well for asteroids and chunks of ice (and has even exceeded expectations in the case of nebulae), but doing so, as you guys have no doubt noticed, gets much harder when one turns attention to ships, stations, and the like without being able to actually draw/reason about them on paper. As I have always asserted of procedural algorithms, though, the failing is not with proceduralism itself, but rather the one doing it. We have but one person to blame for the legendary LT boxships... :oops:

The fruits of this work are already flowing. As of last week, we have a new, joint-based mechanism for procedural algorithms to leverage. Lindsey is working on creating a library of procedural building blocks along with the architectural elements to arrange those building blocks in sensible and aesthetically-pleasing ways. I was particularly pleased when she came to the joint-based solution in her second week of work, since that was an idea that I had been contemplating for the next iteration of better ships and stations. Now I need not contemplate, as the code is happening while I focus my efforts elsewhere. Lindsey will be working over the coming weeks to scale up our procedural geometry toolkit and squeeze as much quality as possible out of proceduralism in LT, especially where it remains lacking. Unlike physics, proceduralism actually is one of the most-advertised and (I like to think?) most-anticipated features :)

I'll leave the fun / detailed bits for her to dive into in her own logs (!!), as I'm sure she can shed more light on converting the artistic process into code.

Note: For personal reasons, Lindsey wishes to go by pseudonym (i.e., Lindsey). Yes, strange, I know...*briefly opens Facebook, sees 40 unread messages & 100+ friend requests from strangers, promptly closes browser and burns laptop* :ghost:



No screenshot gallery for this log, as I don't have much that's visually new to show for the week. In terms of current work, you all now know what Lindsey is up to, but on the other side of the room, Adam is scaling our developer UI up into a full, player-facing UI, since it's currently looking as though LuaJIT may just pull through for us such that UI can remain in Lua (keep your fingers crossed...)! As for me, my focus remains on gameplay logic mechanisms and ensuring performance in the presence of real, representative gameplay logic at large scales (much of which is concerned with computationally-expensive AI that needs to happen frequently but not every frame).

With gameplay logic, UI, and procedural algorithms all getting serious brain-time simultaneously, the months ahead hold quite some promise.

See you next (/this?) week! :wave:




Link to original: viewtopic.php?f=30&t=6340
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: The Limit Theory Email/RSS News Thread

#37
Friday, October 27, 2017

Hi I'm Lindsey

Hello everybody!!! o/// For those of you who don't know, I'm Lindsey Reid, the newest edition to the LT team! Josh announced & explained me joining the team in his last devlog, so read about it there if you haven't already :) TL;DR: Josh and I met (& subsequently started dating) through the Baton Rouge indie dev scene, then I moved to California to work on a game for """the man""", then I moved back here to work with Josh on LT :) I have skills as both an artist and a programmer, so I'm focusing on writing procedural space ships and space stations. I'm starting with stations.

I've been here working on this for a few weeks already, so there's a lot to talk about. I'm also new to writing dev logs for the public eye, so do give me feedback on what you're interested in seeing (other than pictures, I know I know :V) and if I drawl on too long about code and algorithms. OR not long enough!

Soooo... I'm gonna follow Josh's example and first write a brief recap, and then delve into details on some of the fun stuff.

Also, the Imgur gallery is here. Some of it won't make sense without reading the post :V



Brief Recap

  • Researched & drew very, very basic concept art for stations
  • Wrote foundations of Shape library for basic building blocks of procedural geometry, including:
    • Prism (with arbitrary # of sides to the 'base face' & radius, height)
    • Irregular Prism (same as above, but with unequal side length & radius for each side of the 'base face')
    • Ellipsoid (like spheres, but with arbitrary w, l, h)
    • Torus (with arbitrary functions for the inner and outer rings & radius of each)
    • Box (arbitrary w, l, h)
    • Scaffolding (arbitrary w, l, h)
  • Began foundations of Warp library for making those Shapes interesting
  • Wrote foundations of Joint system for combining Shapes along Joints
  • Wrote Piece object for storing combined Shapes
  • Experienced awe and terror at the nebulous challenge ahead of me
  • Pulled art theory out of my butt and made cool shapes with it I guess
(note that I wrote "foundations of" in almost all of these bullet points; while these libraries are pretty solid for now, I fully expect them to expand later as I start building stations with them)



Research & Concept

It is quite the challenge to build a station generator from scratch. Most other games and generators I've seen that leverage procedural content can be broken down into handmade art assets at some level, but we're doing this completely from scratch. Handmade content means you have control over defining an aesthetic down to the details, even if the way you combine them is partially random. But we don't want to hand-make one or two or fives or 100 styles for one or two or five or 100 unique alien ships, we want to be able to generate an aesthetic for an infinite number of alien civilizations. This comes with a handful of challenges that I'll be chronicling my progress in solving with these devlogs. And these problems aren't ones I can sit down and work through in a week- they're questions I've been asking since I started this ambitious project, and I'll be continuing to solve them as I build the codebase and until the day we call this 'done'. I won't really know if I'm solving these problems until I start generating stations and seeing if they look good or not. When they don't look good, I'll re-write and tweak and concept and try again. And when they do look good... we'll have an infinite number of them.

Of course, this is still a game, which means I can't just throw any old random shapes together. Players need consistency and clarity. Most games, even those that use PCG, re-use models, textures, and color schemes for the sake of clarity of their game elements. For example, in any game with treasure chests, even if you're halfway across the game world traversing completely different civilizations, treasure chests all pretty much look the same. But in LT, we want to use PCG to stretch the limits of maintaining recognizability between game elements while encouraging visual diversity between different alien cultures. With stations, for example, it'll be pretty important to be able to look at a station and immediately identify if it's likely to destroy you, say, by the number of visible weapons it has. We can't go crazy and generate just any shape we want and call it a 'gun'- that would be impossible to read. With either clever UI or similarity in shape, texture, or color scheme of weaponry, players need to be able to immediately look at a station armed with 10 huge guns and be like, ah, maybe I shouldn't mess with this guy right now. (or 'ah', let me get out my 10 even BIGGER guns and blow him to smithereens. :V) Then again, it'll be boring (and confusing in its own way) if the high-tech, opulent, flamboyant alien station has guns that look exactly like the guns on the small, scrappy station across the system. Again, we want each alien culture to have their own unique aesthetics. It's a tough balance to maintain. And a challenge I'll enjoy tackling.

My main goal with concepting stations, therefore, much unlike traditional concept art, is not to define exactly what stations and ships will look like, but to identify patterns in how stations are built and why they look good. Over the past few weeks, I've been collecting reference art & photos, both in books and on a Pinterest board, and doodling stations myself. I use these references to identify patterns for how stations are visualized: basic shapes used, clusters and patterns those shapes are arranged in, and overall silhouettes. Those patterns form the basis for how the generation algorithms will work and what shapes, warps, and patterns they'll use to create awesome, diverse stations.

Spoiler:      SHOW
Pictured below: concept doodles for whole stations & for shapes & patterns to build those stations out of. Also, random ideas scribbled on sticky notes, and at the very bottom, art books indexed with sticky tabs for quickly finding my favorite space station concepts.
Image


Shapes, Pieces, & Joints

As an artist, when I look at the world and want to translate it into a drawing, I look for two major things: 1, the composition- the lines of action, negative space, silhouettes, and large shapes holding everything together; and 2, the individual shapes that, combined together, create the composition. I've spent the last few weeks analyzing my own art and other's drawings to figure out the major patterns that can be used to describe these nebulous ideas, and what particular patterns of composition and shapes are used to specifically create space stations. And since I won't have any pieces to generate interesting compositions without the shapes that make them up, I decided to start this whole process by creating a library of shapes.

I chose the shapes to fill the library with by analyzing space station concepts and picking pieces that could conceivably be the foundational shapes that they're all made of. All of these shapes will be warped, combined, rotated, and otherwise modified to create the functional and purely aesthetic modules that make up stations. These are mostly just an estimation of what I need right now, so I'll expand the library as needed.
  • Prism
    • Spoiler:      SHOW
      Image
    • Just imagine the big shape without all of the little rectangles around it, and that's a prism :p it allows for arbitrary side #, radius, and height. The rectangles are actually a demonstration of the Joint system working!!! (which is explained below.)
  • Irregular Prism
    • Spoiler:      SHOW
      Image
    • The code to generate this is actually quite similar to the prism, but the radius and angle for each vertex on the base face is random.
  • Ellipsoid
    • Spoiler:      SHOW
      Image
    • Ellipsoid is just a fancy word for a sphere with arbitrary width, length, and height.
  • Torus
    • Spoiler:      SHOW
      Image
    • I included tori as a primary shape because 1) they're hecking cool and 2) SO many space stations have rings. also, it was fun to code. As you can see in the full Imgur gallery, both the inner and outer ring of the torus use parametric functions to make customizable shapes.
  • Box
    • A box is a box. You know what boxes look like. also i forgot to take a screenshot
  • Scaffolding
    • Spoiler:      SHOW
      Image
    • Scaffolding is kind of an odd one out because it defines a very specific shape. It's one of the first clusters of shapes that I made, and it's really here more just to demonstration that my mesh rotation & translation was working than anything else. I do expect to use it, since scaffolding is such a common shape on many space stations, but it may or may not be one of the 'primary' shapes.

In addition, I started cooking up the library of Warps and Joints. Warps have the ability to apply all kinds of functions to every vertex on a mesh in order to rotate, stretch, skew, and wiggle it into a particular shape or pattern. I don't have fun screenshots or examples just yet because I didn't have time to experiment with this. Instead, Joints were my highest priority this past week. A 'Joint' represents a point on a mesh with a position and a direction. Connecting two meshes on joints, you rotate and translate the first mesh so that the direction on the 1st mesh's joint aligns with the direction on the 2nd mesh's joint. (In math words, you transform the 1st mesh into the basis defined by the 2nd joint.) This is imperative for enabling the station generation algorithms to connect shapes in an interesting way, so it's exciting that we made this happen so quickly. And not only did we get joint connection working, but I also wrote a function that can create joints for meshes that would be difficult to hard-code. The function places a joint at the center of every quad and tri in the mesh, with the joint pointing in the direction of the surface normal. I'm hoping this will turn out to be a useful pattern for the procedural shape-combining algorithms. If it's not, I'll find a way to tweak it. And finally, I created an object for storing combined shapes- that's kinda technical in a boring way, so I won't bother to go into detail. TL;DR: We can now define connection points for shapes and GENERATE connection points for complicated and randomized shapes :)



Fin

Whew!!! I just finished my first ever public devlog!!!!! Whoaa!!! Let me know what y'all think :) And don't forget to look at the Imgur gallery for the full visual goodness of this week!!!




Link to original: viewtopic.php?f=30&t=6343
Ship Inspiration Pinterest!! (send me stuff)

"You’ve got to work on something dangerous. You have to work on something that makes you uncertain. Something that makes you doubt yourself... because it stimulates you to do things you haven’t done before. The whole thing is if you know where you’re going, you’ve gone, as the poet says. And that’s death."
- Stephen Sondheim
Post

Re: The Limit Theory Email/RSS News Thread

#38
Monday, November 6, 2017

Welcome back, bienvenue, aloha, etc etc. You know how this works by now :D Onward!



Recap since Last Time:

  • Moved all heavy physics code (kinematics, collision, attachments/parenting, sleeping) to C (HUGE gains)
  • Added lots of new instrumentation & utilities for profiling; we have a much better handle on CPU time now
  • Major improvements to HDR tonemapping to prevent color desaturation, especially in bright regions
  • Added time acceleration (for testing purposes...)
  • Ported dust-flecks from LTC++; dust clouds to follow shortly
  • Had a lot of fun with Lindsey's new extrusion & stellation functions in our PCG library
  • Added AI aiming / turret control to existing gameplay logic test
  • Stress-tested gameplay script scheduler with great results
  • Loads and loads and loads of fixes, optimizations, cleanups
  • Scripting scripting scripting gameplay logic gameplay logic gameplay logic and such

(I feel compelled to emphasize that seeing these changes in bullet-point format doesn't even come close to conveying the amount of progress made since the last log :) )



Gameplay Logic with Coroutines and Automatic Scheduling

While much of my time over the past few weeks has been spent spelunking in the various caverns of our existing systems in search of milliseconds, it has all been in service to the bigger-picture goal of supporting 'a lot' of gameplay going on at a large scale. Ultimately, although the journey likely won't be complete for yet another week or two, I'm happy to report that things thus far are going very well. In this log, I'll give a more complete picture of what that actually means!

Now, I know already that I'm going to indulge myself here in my technical explanations, particularly since I'm excited about the system at hand. As such, I'll offer a TL;DR for those who don't find my forrays into techno-gibberish-land to be interesting.

TL;DR: It's now very easy to write gameplay logic that can run efficiently at huge scales in the midst of lots of other logic. Our systems take care of figuring out how, when, and which logic to run each frame to achieve maximal accuracy alongside maximal performance. While the system can support a tremendous amount of logic going on at once without hurting performance, it can also automatically and gracefully degrade the accuracy of simulation logic in order to maintain FPS when 'tremendous amount' turns into 'put out the fire on my CPU!!'

Those who enjoy gibberish, proceed, those who don't, skip to next horizontal separator :)

I've given a few code examples in the past of mod format, CType usage, Lua in LT, and so forth. What I haven't given is a look into how 'real' gameplay logic will look under our work-in-progress gameplay logic system (which is, for the moment, rather unimaginatively named the 'Script Scheduler'). Time to change that! How is it that we will develop that which will go on to bring life to LT? Let's have a look!

Meet my 'simple AI test' script:

Image

Clearly I've left out a lot -- I've folded the initialization, which just sets up some state for the AI unit and is largely uninteresting. I've omitted definitions for the two functions that seem to be doing the work here: aiManeuvering and aiTurretControl. All I've left is the outer body, but that's because the interesting thing about this code is not so much the content, but the structure. If you're code-savvy, you may have already picked up on the bizarre elegance of what's going on here: the script itself -- which is, at this point in testing, the only thing giving AI units any behavior at all -- is merely a function! Not only is it just a function, it's a rather bizarre one at that. It initializes some data and then goes into a seemingly-infinite loop (the 'while' block). There's precious little to indicate that it's part of a larger system. In fact, from the standpoint of the one writing this code, it almost looks like this is the only logic in existence! The code seems to completely deny being part of an ecosystem of gameplay logic. It is not worrying about keeping track of some 'state machine;' it's not defining a bunch of callback functions to interact with the outside world; aside from local variables there's virtually no persistent state of any kind! Overally, it looks nothing like what one would expect from a gameplay script.

Remember a while back when I said something about the needs of gameplay logic being very similar to the needs of a process in an operating system? :) That is, in fact, almost exactly what's going on here! Our current implementation of the script manager functions similarly to an operating system scheduler, and gameplay logic scripts function similarly to a process. Indulge me in my desire to explain how and why this is beautiful for both the one writing code and the system running it :nerd:

I've expressed over and over since my performance failures in LTC++ that exploiting sparsity is key to being able to support a thousand AI ships running low-level manuevering logic, while simultaneously having research modules all over the universe working toward technology breakthroughs, while simultaneously letting AI players in high places run cost-benefit analyses of hundreds of potential projects, while simultaneously simulating the black-box economies of colonies everywhere, while...you get the idea! Recognizing that these calculations each have their own unique demand patterns for CPU cycles, then being able to predict and accordingly exploit those patterns...well, this is the difference between LT and solitaire :ghost: But doing so while also making it tractable to write all that logic? I had a name for the problem: FPLT, the fundamental problem of LT. At the time, someone rightly pointed out that it was not so much the fundamental problem of Limit Theory so much as it was the fundamental problem of simulating any large, complex system in real-time -- a problem encountered by many others at many points in the past.

Operating systems get closer to solving that problem than any other piece of software known to me. Some solve it better than others... :ghost: An operating system must cope with: an unknown amount of work needing to be performed by an unknown number of workers at unknown time intervals, interacting with one another in unknown ways, and having an unknown number of CPU cycles to spend each unit of time to accomplish all of this. By unknown, I mean unknown beforehand -- unknown at the time of building the operating system. Hence, a good OS must be prepared to efficiently support an arbitrarily-complex web of processes using a finite amount of computational power. It must also do so in such a way that the burden is not overwhelming on the programmer when he/she needs to write a program that performs such work. If you really think about it, this is quite an amazing feat! Say what you will, but modern OSs are indeed pretty good at this central job of theirs. How?? Chiefly, via two mechanisms: intelligent scheduling and virtualization of resources. The former attempts to maximize resource utilization such that the most work-per-unit time is achieved (over both the short- and long- term). The latter makes it appear to programs that they have exclusive control over and unlimited usage of each resource, in order to ease the burden of programming. The most important resource in question is, of course, CPU time.

Enough of that, let's get back to our AI script and understand what's really going on. Clearly, something's up with that 'sleep' function. Indeed, it is central to how our script mechanism works. Under the hood, we're using Lua coroutines, a somewhat advanced programming concept to fully understand, but one with simple enough implications: scripts can run as though they were their own 'processes,' living within a function (or a multitude thereof), while still being able to give control back to the rest of the game so as not to forever stall the universe while one silly pilot tries to calculate a successful womp-rat-bullseye. Now we see what's actually happening here: those calls to sleep are 'giving up' (in coroutine parlance, 'yielding') control of the CPU to whatever thing gave it in the first place (here, the script manager). In the world of coroutines, one thinks less of code like this:

Code: Select all

  other code other code other code
    other other other
    ...
    <--  I am here :'[  -->
    ...
  ...

And more like this:

Code: Select all

  my code my code my code
  ...
  <-- yield; everyone ELSE is here >:] -->
  ...
  mine mine mine mine mine

This so-called inversion does wonders for being able to reason about game logic! We no longer have to consider the context in which our code is running -- much like a process in an OS need not worry about what else is actually going on with memory or the CPU (multithreading aside...). We simply write the entirety of what we want our game logic to do, from start to finish, and, wherever it makes the most sense (usually inside a looping construct), we will insert these 'sleep' calls to allow the rest of the world to do what it wants.

But that's not quite the end of the story, because this code is not really leveraging our system to the fullest -- it's being intentionally greedy in order to stress-test the script manager. As you might have guessed, that number inside the sleep call is a number that indicates how long the script wants to sleep. Sleep(0) essentially means "look, I'm going to let everybody else have some time to do their thing, but WAKE ME UP AS SOON AS YOU CAN." There is no 'sparsity' being exploited here. Let's consider a different script, one for a manufacturing module's logic, which is almost too good of an example for exploiting sparsity:

Image

This one may have mistakes since I just cooked it up, but it's still very much representative of what writing LT logic will look like. Manufacturing here is dead-simple. Suppose we have a 'job' description (it will come from the blueprint of whatever's being manufactured) that includes some inputs (materials consumed), outputs (materials produced), and a time indicating how long a single run of the given job takes. All we do in our script is check that our inventory contains all of the required materials to run the job, then remove the prescribed amount of input material from inventory, call sleep with the given job time, and, upon waking, feed the outputs into inventory. We can easily put this inside a loop so as to allow instructing manufacturing units to continue manufacturing until some prerequisite number of runs have been completed (or materials run out).

Here we have a true example of something that will benefit immensely from sparsity. The vast majority of the time, manufacturing logic consists of precisely nothing. If you were to write this in 'onUpdate' style, you would have a state variable, a switch or if/elseif/else sequence checking the state, and, within one of those branches, you would simply be checking if elapsedTime >= job.batchTime (which would be false for 99.9% of the time, until the final frame where the job completes). Now, run a million manufacturing jobs accross the universe at the same time, and what will happen? The other style will bog down. Maybe it will take more than a million. But it will come to a grinding halt at some point. Our version with sleep? You will see no noticeable difference in performance until the number of jobs gets so large that it starts forcing other game memory out of RAM and into swap space. It will be long, long after the other system has come to a screeching halt. This is because, with a good scheduler, we need exactly no time at all to process sleeping jobs (this is a tiny lie, but only tiny -- asymptotically, I am not lying, because sleeping jobs require O(1) time for the whole lot of them).

I have shown two examples at the extremes -- low-level AI maneuvering isn't a great example of sparsity, while manufacturing is a perfect one. But consider that LT logic spans the whole spectrum in-between, and that, by my estimation, that spectrum is heavily-biased toward the sparse side. What we have here is indeed the game-changer that we were looking for with respect to managing logic of all granularities. Huzzah! \o/ :squirrel:

Closing Notes:
  • Logic is never scheduled before requested, but may be scheduled after. For this reason, sleep returns 'delta time' -- the actual amount of time elapsed since the script yielded. AI uses this, for example, in the PID algorithm, which requires knowing elapsed time. The manufacturing script has no need for the true dt, though it does operate under the assumption that at least job.batchTime time will have elapsed when it is awoken (our scheduler, like most, guarantees an 'at least' relationship with respect to sleeping)
  • The entirety of script execution can be bounded such that total script time is never allowed to take, for example, more than 10ms per frame. Doing so allows maintaining performance -- but our scheduling algorithm also ensures that each script will still be called with an importance that is proportional to how long it's been since the script should have awoke
  • Due to the above 'fairness' policy, dt becomes 'larger than expected' for each script at roughly the same rate,
    meaning that each script's temporal accuracy degrades at roughly the same rate, so the simulation accuracy degrades smoothly and fairly in the face of too much work
  • Since the scheduler is deeply concerned with time, it is very easy to profile the total CPU time of each script! This allows much more insightful information than just 'Ship.onUpdate' or 'AI.onUpdate,' as it was in LTC++ profilers. Now we will actually see accurate time consumption for each piece of logic, aggregated over all entities on which that logic is running!
  • Speaking of which, even in its infancy, scheduler has been profiled with tens-of-thousands of scripts running at the same time with no degradation
  • The future of gameplay logic is BRIGHT! :D



Whew. Took two-and-a-half turns of the hourglass that time...but I regret nothing! It has been such a long road to this system. As I mentioned above, it'll still be a week or two before the whole thing is finalized. But things are shaping up really nicely, both from the performance standpoint and from the ease-of-use standpoint.

It's a shame that I don't have more time to talk about the tremendously-exciting developments on Friday, what with Lindsey's implementation of extrusion and stellation for station-building, plus the addition of the icosahedron primitive. I did, at least, take an excessive number of screenshots while playing with the additions. So...gallery! Note that any lack of diversity here is not the fault of Lindsey, as our primitives library is now quite large, but is to blame on my obsession with icosahedra and their beautiful symmetry...which made it hard for me to focus on any other shape :squirrel: There are also a few AI unit shots and nebula shots in there, just because :)

I must now turn attention to the KS update! See you next time ;)



Limit Theory Devlog Gallery ~ November 6, 2017

A few of my favorites from the gallery, showcasing some very basic results using almost nothing but icosahedra, stellation, extrusion, and warping (the occassional torus does show up). Thanks, Lindsey!

Image

Image

Image

Image




Link to original: viewtopic.php?f=30&t=6356
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: The Limit Theory Email/RSS News Thread

#39
Hello again! We have two devlog updates that have happened within the past few days, so I decided to post them both at the same time. The first one is from Lindsey, and the second one is from Josh. Enjoy!



Friday, December 1, 2017
Originally posted by LindseyReid
~~~~~~~~~~


Intro

First of all, thank you for everybody (espeically our potentially confused non-American forum-goers :V) for being patient while we all enjoyed our Thanksgiving break ^^ Since more holidays are coming up again soon, I'll be posting one more devlog this year after this post. My last devlog of 2017 will be two weeks from today, on Friday, December 15th. After that, my devlogs will resume again on a normal two-week schedule on Friday, January 12th.

Josh and Adam don't have exact devlog plans like this, but I would expect them to be similarly interrupted by the holidays. Although I can tell you that y'all will probably be quite pleased by the progress they've been making during this devlog silence. ;00 We always appreciate your patience <3

If you need a daily shot of LT, you can follow me on Twitter, where I post screenshots almost daily. In addition I'll also link every LT tweet on these devlog threads. That way, those of you who don't have a Twitter can check back here for those screenshots :) Although the tweets will be more frequent than devlogs, expect my LT-related tweeting to be sparse around Christmas through New Year's Eve.

Speaking of screenshots...

Limit Theory Devlog Gallery - Lindsey 12.01.17

Now, on with the words!!!


~~~~~~~~~~


Summary
11.10.17 - 12.01.17

  • Added more data to Joints
    • Added up vector
    • Added scale
    • Added function to generate a JointField from a Shape
  • Added AABB function to Shape
  • Cleaned up all basic Shape functions
    • Ensured all shapes are manifold
    • Ensured no basic shape functions create bad polys
  • NEW SHAPE: 5-faced pyramid
  • NEW WARP: Bevel
  • NEW WARP: Smarter tessellation
    • Refactored tessellate() to tesselate quads INTO quads
    • Refactored tessellate() to produce 0 duplicate verticies
  • NEW WARP: Greeble!!!
  • Implemented a policy for protecting against crashes & bad geometry as a result of trying to apply a warp on a bad poly.
  • Began work on Style

~~~~~~~~~~


Boring Architecture Stuff

Joints, as y'all remember, are used to define patterns of shapes and create clusters of shapes. I added two crucial pieces of data to this to really bring out the strength of Joints: a Scale and an Up vector. Joints now store Position, Direction, Up, and Scale. (If you're familiar with Unity, they're basically Transforms at this point that don't have any other components like a mesh.) The Scale allows the user to define how a Shape will grow or shrink in size when placed on the Joint. The Up vector is a mathy thing that gives more strength (and prevents errors) when using the Direction property to define what direction the shape will face in when placed on the Joint. To complement this, I made sure that all of the basic Shape creation functions (like ellipsoid, box, and icosahedron) as plain as possible. They just give you a shape, and require no input regarding rotation, translation, or scale, as all of that is handled by Joints now. I also ensured that all basic shape functions return a shape that fits inside a (1,1,1) box, and added an AABB function to get size information about shapes. All of this work combined means that we now have more intelligent ways to combine shapes based not only on some abstract position, but also on a position relative to a shape's scale.

Here's a small demonstration of AABBs & the new scaling system working. The scaffolding blocks to the left of the sphere are using the sphere's AABB information to be placed perfectly at the edge of the sphere and span the length of the sphere.
Spoiler:      SHOW
Image

Here are two demonstrations of the new Joint properties. Both images use the function to generate JointFields from a shape; they generated a Joint for every quad on the large box, then added smaller boxes to every one of those joints. The second one demonstrates using the Joint's new Scale property to scale the size of each smaller box based on its Y-location on the larger box.
Spoiler:      SHOW
Image
Image

I also spent a day ensuring that every basic shape function produced perfect data. Specifically, I eliminated any 'bad' polys and ensured all of the shapes are manifold. (A 'bad' poly is defined here as having a zero-length normal and/or fewer than 3 verticies.) The Irregular Prism was creating one bad poly, so I got rid of it. The Box was non-manifold, so I re-wrote it to be manifold. For the purposes of fixing the Box, the mesh being 'manifold' means that there are no duplicate verticies with the same location. Put another way, two indicies in a poly may refer to the same vertex, but all verticies with a unique position have only one listing in the Vertex list. This isn't the complete definition of manifold, just to be clear!!!, but it worked for the purposes of fixing the Box algorithm.

After ensuring that no basic shapes generated bad data, I implemented a new policy for protecting against attempting to warp invalid polys. This is a bit of a tricky situation, because while I can guaruntee that the original input to a slew of warping functions is perfect, I fear that our infinite universe with its random combination of warps could still create bad polys. For example, tessellations greatly reduce the size of each poly every time they're applied, since you're splitting one poly into multiple pieces. This could theoretically create a situation where a triangle became narrow enough that it was essentially a line, and therefore had a 0-length normal. So, for the purpose of preventing crashes & junk shapes in release, we SKIP performing any operations on invalid polys. From the testing I did, this 1) creates shapes that only have 'good' polys operated on, which still look good, and 2) prevents potential crashes and ugliness from the spreading of NANs and other unwanted mesh data. I'm going to continue thinking about this problem further, as it's a rather dry, but still suuper-important one in the world of limitless procedural geometry.

Before the break, I had begun working on Style. Style is the object to generate and store all of the properties that make a faction's aesthetic unique. Honestly, I'm not super happy with how my first implementation of Style worked out. I think my biggest struggle with it is that I don't have a good foundation to test it on yet. :lol: It'll be more useful to start iterating on Style when I have at least one complete station part or one complete ship type to work on. Before Thanksgiving break, I had been working on building station parts from a perspective of starting with extreme randomness, and THEN trying to refine it. Josh had also previously tried this method, and we both found it to be ineffective. Instead, next time I attempt building functional-looking parts, I'm going to start with a low randomness/ high functionality algorithm and THEN stretch the algorithms until I reach the limits of what still looks recognizable. I believe that I'll have much better results with both making recognizable parts AND iterating on the Style architecture if I come from that perspective.


~~~~~~~~~~


Fun Shapes & Warp Stuff

I added a 5-faced pyramid shape. I also contemplated n-faced pyramids, which would also allow cones, but then got distracted by something else I think. Now that I'm here writing about it, I'll probably do that tonight & update y'all later. LOL.

Spoiler:      SHOW
Image


Josh, in his spare time, wrote a bevel function, because, of course, Josh. It's quite incredible- it can take ANY shape and round ALL of the edges by an arbitrary amount. The 'amount' is a number between 0-1 which describes how rounded the corners will be- 0 leaves them sharp, and 1 rounds them all the way to the next edge.

Here's a bevel on a basic cube.
Spoiler:      SHOW
Image

The same cube, with a higher amount of bevel.
Spoiler:      SHOW
Image

The same cube, with a bevel amount of 1.0.
Spoiler:      SHOW
Image

A bevel on a cube whose faces have been extruded.
Spoiler:      SHOW
Image

A prism that's been beveled, extruded, and then beveled again.
Spoiler:      SHOW
Image


I added a few different variants on tesselation, and generally made the algorithms & their uses smarter. We now have several options for tesselation & triangulation:
  • A fan-shaped triangulation, which creates 0 new verticies. This doesn't preserve the angles of the original poly, and thus is currently only applied at the last step of shape creation, right before the shape is converted into an engine-level mesh, which is used for rendering.
  • A centroid-based triangulation, which doesn't preserve angles, and is used to break up polys with >3 indicies.
  • A triforce-shaped triangulation, which preserves the angles of the original poly. Currently, it can only be applied to tris, so right now a centroid-based triangulation is applied to any polys with >3 indicies before applying the triforce triangulation. This is great for creating detail is meshes before applying warps like greeble().
  • A tesselation for quads that splits them into smaller quads. A quad is a poly with 4 indicies- basically a rectangle or square. Tessellating quads into more quads preserves the original intention of the poly better than splitting it into tris. It's used in our shape:tessellate() algorithm in conjuction with the triforce-shaped triangulation to create more detail on shapes before applying ceratin warps.

Here's a hand-drawn picture from yours truly to show what the different triangulations look like:
Spoiler:      SHOW
Image


Aaaaand finally, I wrote a Greeble function!!! w00t!!! Greebles are small, repeated details added to the surface of a mesh to give it the appearance of detail and scale. For a classic example, think Star Wars ships:
Spoiler:      SHOW
Image

Here's a box, in LT, beveled, extruded, then greebled:
Spoiler:      SHOW
Image

And here's a long, greebled tunnel:
Spoiler:      SHOW
Image


~~~~~~~~~~


FIN

My plan moving forward is quite exciting: between now and the new year, I want to jam on generating a complete bomber-sized ship. I feel pretty good about the library of shapes, warps, and tools to use and combine them that I have built up. Previously, I wanted to work on station parts first, but I hit a wall with them because of their complexity. Station parts need to look extremely functional, which leaves little room for experimentation and a lot of room to get frustrated about whether something looks 'real' enough or not. I can leave that frustration for later, once I have a grasp on generating something small & have built a more robust Style system. A small ship would just be more exciting and a bit simpler. In addition, this would give me a more solid base to work on to iterate on the Style implementation.

Expect exciting devlogs coming up! Thank y'all so much for joining me in the 3rd devlog of my LT journey. Remember that I've got one last devlog after this until 2018! But you can always catch me on the forums, Twitter, or the IRC in the meantime :)

Gallery link again, because DUH.

Thank you,
Lindsey Reid

P.S.: for anybody who enjoys my blog tutorials - I WILL be writing a greebling tutorial & a triangulation tutorial soon! Let me know in the comments which of those or other topics you'd like to see.




Link to original: viewtopic.php?f=30&t=6380



Monday, December 4, 2017
Originally posted by Josh Parnell

Hey again everyone! Hope you all had a good Thanksgiving (for those to whom it is applicable). I certainly ate my fair share :ghost:

It's been a productive few weeks since the last log (minus Thanksgiving break), but last week, in particular, saw two major breakthroughs that have me really happy with the current state of affairs :)


~~~~~~~~~~

Major Progress Since Last Time:

  • Major breakthrough with being able to use Lua data & functions from our C subsystems
  • Used the above to implement a C-side scheduler that schedules Lua logic for execution; significantly faster than previous implementation
  • Ported planets & atmospheres
  • Integrated Loom for advanced inspection of LuaJIT traces and assembly
  • Implemented several new algorithms for procedural geometry library, touched on in Lindsey's latest log
  • Loads of quality-of-life improvements to how 'game-level' Lua is written
  • First-pass design of mod format taking into account all existing support systems

~~~~~~~~~~


We're All Modders: Hammering out LT Mod Format for Building Gameplay

At this point, we've essentially got all of our core support systems working. By that, I mean that we have either Lua-level or C-level (in some cases a mixture) mechanisms for supporting almost everything gameplay code could want to do: from defining new datatypes that can be used with the utmost efficiency, to adding and/or modifying existing ones, to writing gameplay code that needs to run in response to certain events, or needs to be scheduled to run at certain frequencies, to writing new methods that can be called on the aforementioned datatypes, to defining jobs that batch-process large chunks of entities that match a certain filter, and so on. Where there was once a wild-west of Lua trickery, there is now a solid set of production-ready, battle-tested system. So, what remains? Naught but the orchestration of said systems to compose the symphony of gameplay that is Limit Theory :)

So, it is time that we once again turn our attention to the format for this orchestration: the Limit Theory Mod Format. I have expressed many times that I wish for our vanilla LT gameplay-level logic to be written in exactly the same manner as any other mod, and I'd like to stand by that -- both for the sake of making it easier on us devs (since we'll have a well-defined, decentralized framework for incrementally adding modular bits to the game), as well as for the sake of ensuring that LT is not just moddable in theory, but is built on the very proof that it's moddable in practice.

Thus far, my solution is unfolding as a two-part mod format, where each part may exist zero or more times within any given mod file. One part is data, the other part is code. Data sections use a very simple 'data description language' to perform the aforementioned orchestration of our system. A data section for implementing an inventory might look like this:

Image

Hopefully it doesn't take too much brainpower to figure out what's going on here: we've opened up some object scope called 'Ship,' declared that it has an Inventory, and then gone on to define Inventory itself. Inside Inventory, we can see three types of 'declarations,' each of which corresponds concretely to a programming concept and each of which will be used in a different way to feed our underlying systems to build the required functionality.

As the programmers among you can likely guess, 'has' declares a field/member variable, 'can' declares a member function, and 'when' declares an event listener (a piece of code that will run in response to a particular 'event' that may occur on the object). The syntax of it all is just a sketch, so let's not concern ourselves too much with quibbling over whether 'has/can/when' is too cute / whether semicolons are the bane of mankind, etc :ghost:

The point is that we have a portion of the mod that's using a very simple 'data' language to inform all of our heavy-duty systems of how to make inventories work. The 'has' statements inform our CTypes system about fields, address, alignments, etc. The 'can' is informing our more-general type system about the presence of callable member functions. The 'when's are informing our event handler system about events and their listeners.

Then, of course, we must write some Lua to make it all work:

Image

And this looks more like what you would expect from standard Lua code. Here we're just implementing the functions referenced in the data section with standard Lua + some of the extra features added by the LT engine. Easy! All the 'bigger picture' elements that dictate when, where, and how a particular function runs are specified in the data section (e.g. addItem will be a method that can be called on any object with 'has Inventory', so that in other code we may say myShip:addItem(DenseTalvienite, 17) for example). Hence, much of the need for a modder to 'worry about' the internal mechanisms used to make LT performant (you know, the ones I talk about in pretty much every devlog ever?) is alleviated. Granted, they still exist and may still be interacted with at the code level, but for the majority of gameplay functionality, doing so should be unnecessary.

The Lua mod loader that parses all the mods into a format that can then be shipped off to the appropriate engine-level systems is already implemented; what remains is to do the shipping-off bit and to move existing gameplay code into isolated mod files where appropriate. I'd like to have this done by the end of this week :D

In response to anticipated concerns about 'oh no another language / LTSL / JOSH BE CAREFUL' :roll: : Rest assured, the goal is entirely different here. I didn't define a new scripting language; this is just a very simple data-description language that helps us connect all the moving pieces of our mod! It can be parsed with ~20 lines of Lua and the lovely string.gsub function. Anything more complex is unnecessary. I am not rabbitholing, I promise :)


~~~~~~~~~~


Side-Soliloquy: Why So Much Concern for Moddability?

This bonus Josh-soliloquy is an optional part of your devlog experience :lol:

Spoiler:      SHOW
Many of my technical expositions, and even the less-technical ones, are ridden with talk of moddability and concern that a given system will not hinder the extensibility of LT. I feel that it might be a good time to step back and look at the big-picture of LT development for a moment; to express what modding means to me, what it has meant for Limit Theory's development process, and how it has impacted both to the utmost degree. To open, I'm going to make a daring assertion: I believe that Limit Theory would already have been released by this point had I chosen to stick with my original vision of 'no modding.' Take it as you will, perhaps noting that I do have a patently lackluster track record of timeline judgement (:oops:) Still, I feel that there is quite some truth to the statement. After all, we had the solid beginnings of a game with the release of the LT Prototype all the way back in April 2013 (!!)

Sometime in 2014, the tides began to change. I began to recognize the need for an external data format in which to place 'arbitrary' constants like those relating to high-level balancing (how much energy is required to negate one unit of damage compared to dealing one unit of damage? How much energy is required to extract one energy unit's worth of raw material from an asteroid? etc..) Naturally, it wasn't long before a similar need for 'arbitrary' code was recognized. The solution at the time was LTSL, and, as we all know, thus began the march toward the dark days, FPLT, and some of the hardest months of my life.

Looking back, it's clear that my solutions, from that point on, were guided by an unspoken requirement: LT had to be moddable, on that point I was no longer willing to yield -- else I would surely have eventually come back to C/C++, learned to write a more modular engine with faster compile times, and found a way to iterate on gameplay code without needing a separate language. It's hard to admit. But it's the truth. Hence, today, we are in a world where great sacrifices in development time and effort have been spent on Limit Theory's touted modding capabilities. Personally, I'm OK with that. And here's why.

When others ask about LT, moddability often comes up. Invariably, I bring up Limit Theory's suggestions sub-forums at some point, typically bragging on it as being "surely on of the best places on the internet to find inspiring, space-game/sci-fi ideas of remarkable breadth and depth, elaborated on by people of singular imagination and intellect." Since the KS closed at the end of 2012, there have been >1,000 topics and >20,000 posts in this subforum alone. It is a remarkable place -- one that I wish I had the time to read front-to-back. But that's just it: I don't. I don't even have enough time to read all of the beautiful, elaborately-thought-out gameplay suggestions generated by you all, much less implement them.

What I can do? What I'm really good at doing, as this whole process has taught me, and perhaps, what it is that I love doing above all else: it's probably not game design, it's probably not even procedural nebula algorithms :ghost: ...no...building things that allow other people to realize creation with ease, simplicity, and power...now that is the kind of thing that can get me out of bed in the morning. And that's exactly what I/we have done. We've built something that can take a high-level, simple description of a new gameplay mechanism, and do everything that needs to be done behind-the-scenes to let that mechanism play nicely with a thousand others, in the presences of tens of thousands of entities stretched accross a boundless universe. We've built something that renders many of your suggestions doable in a few hundred lines of Lua or less. And THAT...I'm proud of that :)

Everyone knows that Morrowind was one of my favorite games of all time. And it still is. It's still cranking, 15 years after release (in 2002). People still play it, people still buy it, people still love it. All because Bethesda decided they would build their game with a Construction Set rather than pure code, thereby enabling others to add to their original vision (which was already fantastic) via that tool. And the result is that today, some of us are still talking about how stunning Morrowind looks with graphics overhaul 3.0 et al; some of us are still in awe that Tamriel Rebuilt is alive-and-kicking, expanding the vanilla game's playable area by god-only-knows how many square miles.

And this is my dream for Limit Theory: that it ships as the vision you all supported in 2012 + the modding capabilities that have since become a core feature, and that, in 10 years' time, it has become something utterly and totally beyond that which I could have done myself or with our small team in the same amount of time. I really can't wait :)


~~~~~~~~~~


I've left out all talk of a 'breakthrough' that happened last week involving Lua data playing nicely with our C systems. It's technical and I've decided to err on the side of not-so-technical this week, but, briefly, I'll just say that this breakthrough has opened up a lot of fantastic augmentations to our existing systems, as well as tremendous new flexibility in splitting data & logic between Lua and C (using whichever makes the most sense for the task at hand). Previously, Lua->C interop was easy. LuaJIT's FFI makes it basically effortless. But going the other way? Accessing Lua-side data / calling Lua-side functions from C? It was not so easy when that data/function could be arbitrary and needed to live alongside supporting C data (think: sparse logic scheduler). Since Lua is GC'd, we can't get pointers to Lua data, and this is a major annoyance about which I've ranted before. However, there's a function that I overlooked until last week...hidden in the Auxiliary Library API...luaL_ref...and it solves all my problems. *Slams face vigorously for having overlooked this gem.* At least I know now. :shifty:

Until the 'morrow! (Well, actually I plan to do my next log on Friday. If I'm 'close but not quite' with respect to the mod system, then it will be Monday. Stay tuned!)

:wave:

PS ~ Sorry for lack of screenies, but most of this work wasn't amenable to it. Besides, you guys just got a whole boatload of them in Lindsey's last log! ;)




Link to original: viewtopic.php?f=30&t=6385
Post

Re: The Limit Theory Email/RSS News Thread

#40
Friday, December 15, 2017
If y'all remember from last devlog, I promised that:
LindseyReid wrote:
Fri Dec 01, 2017 7:41 pm
My plan moving forward is quite exciting: between now and the new year, I want to jam on generating a complete bomber-sized ship.
So, I have good news and bad news! The bad news is that I was out sick much of last week with a nasty post-holiday-cold, so I wasn't super productive then. The good news is that I was able to bounce back this week with MAJOR breakthroughs on ships!!!

Forgive me, this devlog is going to be very photo-heavy and very word-lacking. I'll come in tomorrow and add more words to describe the technical stuff in a little more detail, but I wanted to make sure to fulfill the promise of devlogging today. I hope that's ok with y'all ^^

I started the whole ship-building process by gathering tons of reference material. Josh has a small library of sci-fi art books that I scoured and marked pages for inspiration. I also created a Pinterest board of fighter ships, which I filled with my favorite art from Artstation and wikis for games like Eve and Freelancer. I then spent a short amount of time sketching on paper what an extremely basic fighter ship looks like, and broke it down into its basic shapes and parts.

Spoiler:      SHOW
Image

I then built that extremely basic ship in code. Given this base, I then started adding randomness to it and tweaking the min, max, and average values for all of the bits of randomness. The randomized variables on these ships include wing position, wing rotation, wing size, number of wings, body shape, body size, warps applied, and more. Half of my work is coming up with boundaries to push as far as randomness goes, and the other half is tweaking the parameters for the randomness so that every ship has the chance to look unique, while still coherent.

The algorithm that I ended up with at the end of the week is pretty different from the one I started with, and is much more likely to create crazy-looking ships. After some discussion with Josh, I decided to separate the fighter algorithm into two: one that makes tame ships, using code from earlier in the week, and one that makes crazier ships, using later code.

Here are the three Imgur galleries of ships that I generated over the week.

LT Ships 12.13.17
LT Ships 12.14.17
LT Ships 12.15.17


I hope y'all are liking these ships so far!! ^^ DO give me feedback on them. I'm curious to see what y'all think, especially shapes, patterns, and ships that you want to see that the algorithms aren't currently generating. Feel free to send me things here on the forums (I mainly stick to the devlog threads), on the IRC, on Pinterest, or on Twitter @thelindseyreid. Inspiration is a huge part of this process. ^^

Thank you,
Lindsey Reid




Link to original: viewtopic.php?f=30&t=6399
Ship Inspiration Pinterest!! (send me stuff)

"You’ve got to work on something dangerous. You have to work on something that makes you uncertain. Something that makes you doubt yourself... because it stimulates you to do things you haven’t done before. The whole thing is if you know where you’re going, you’ve gone, as the poet says. And that’s death."
- Stephen Sondheim
Post

Re: The Limit Theory Email/RSS News Thread

#41
Hello! It's Friday again. I've tried to keep some form of semi-constant updates coming on Fridays on the forums, so I'm posting here with the most recent news and developments. Our most important recent news is that LT will be at PAX South this year, in just over two weeks! We had planned something like this before, but the confirmation from the Louisiana Tech Park only came around a week ago. Josh and Co. are hard at work getting something ready and polished that we can demo. :) Because of this, and because of the Christmas season, there will not be an LT devlog today.

In case anyone missed it, here's the PAX South announcement, and a link.
JoshParnell wrote:
Wed Dec 20, 2017 12:57 pm
Limit Theory @ PAX South 2018

Hey everyone!

I'm going to be hopping on the devlog train in a moment here, but before I do, I want to get this information out ASAP so that anyone interested can make arrangements sooner rather than later. And the big news is...already spoiled by the title of this post!

We'll be demoing LT again at PAX South this year! When I say we, I mean all three of us from the office here will be going, so if you'd like to praise Adam for his stupidly-fast BSPs, or Lindsey for her stupidly-fast progress on great-looking ships, or me on my stupidly-lackluster ability to meet deadlines :ghost: ...you can do any or all of the above!

PAX South will be held in San Antonio, TX and will run from Friday, January 12 to Sunday, January 14. We'll be there with an LT demo all three days, although at the moment, it appears that we may be closing up early on Sunday. I'm waiting for some clarification from the Tech Park on our Sunday schedule, and will update this post & thread as soon as I know for sure. So, if you're interested/able to come, pick whichever is most convenient for you, but don't bank on us being around after ~12ish on Sunday. The cost is $40 USD for a one-day pass. Unfortunately, it appears that the three-day passes are already sold out.

Official PAX South Site Links:
Registration & Purchasing a Pass
Full Schedule

As with last year, we're going as a group with the Louisiana Tech Park, so to find us you should look for some kind of signage related to "Louisiana Tech Park," "Baton Rouge Game Developers," or possibly "New Orleans Game Developers"...and of course, we will have a smaller sign with "Limit Theory" on it. Our setup will likely be a mounted HD TV with a gamepad, since we'll be pressed for space.



The Demo

As of this week we've started planning out and sprinting on features for the LT PAX demo. Naturally, since LT isn't ready to play yet (nor would the full game make for a compelling five-minute demo experience), our demo will aim to give 'a taste' of Limit Theory in a format that's more amenable to picking up a gamepad for a few minutes and having some fun. It will be unequivocally superior to last year's "war of the box 'ships' demo."

If all goes according to plan, we'll be showing off a combat demo wherein you must defend against progressively-more-difficult waves of attacking ships. Successful defense rewards the player with credits that can be used to purchase upgrades to weaponry, consumable items, full-ship upgrades, new AI teammates, and possibly more :) All of this will, of course, be happening in procedurally-generated systems with procedural asteroid fields, planets, nebulae, and (possibly) stations (although they may be of the 'old' variety since Lindsey is focused on ships at the moment).

Additionally, since Lindsey has made great progress with ships, we'll be integrating a 'ship generator UI' that allows for customization of the generating parameters, so that you can play around with customizing your ship and seeing the PCG work in real-time! She's got some great ideas concerning how we might even save the best user-generated ships so that they continue to pop up throughout the day (you may come back to find someone piloting or fighting a ship that you custom-generated!)

If you've got reasonable ideas/feature requests for the demo that you think would be fun for people to see, please post in the official LT PAX Demo Suggestions thread, but please read the first post before doing so!



If you're able to make it, then we look forward to meeting you! Otherwise, fret not, for there will surely be an abundance of shinies from the ordeal, including an endless fountain of screenies, some real-life pictures so you can see just how photogenic we programmers are, and hopefully some video capture of user play sessions.

:squirrel:
If you'd like to log in and post suggestions for what they can include in the demo, Josh posted a Limit Theory PAX Demo 2018 Suggestions Thread.


On a different note, a number of you brought up concerns that we were cherry-picking which fighters we showed from the new algorithm. To address this, I threw together a short video with unedited footage from the LT shipgen sandbox, cycling through a number of real-time rendered procgen ships. Important note: this algorithm is still a work in progress! The texturing needs done, and there may be a few other tweaks that need to be done here and there, but for the most part the team agrees that as far as fighters go, they look pretty solid. :) There are no skipped ships here, so this gives a fairly good indication of the current average quality.

Here's a video for those of you that browse the forum:



I'm running it on a Nvidia 1050Ti mobile graphics card. FPS stays over 300 in most cases, sometimes going as high as 400. I accidentally ran it from my laptop's Intel integrated graphics chip at first, and it still managed a consistent 60fps without recording. In short: performance is currently outstanding. I don't think Josh has any doubts now as to whether his engine is powerful enough to get LT running.


That's all the news for now! I hope all of you have a good Christmas season, and a happy New Year's! Thanks for staying tuned, and I'll continue to do my very best to keep everyone updated. :)
Post

Re: The Limit Theory Email/RSS News Thread

#42
((Note from Talvieno: I apologize for being late in posting this and I'll make sure I post these more promptly in the future. I've been without Internet for over a week, but things are more stable now.))

Friday, January 5, 2018

Intro

Let's talk about ships!!!! In this devlog, I'm going to explain the general process for how I create the ships, both creatively and (to a lesser extent) in code. I'm going to link this devlog to anybody who asks me "how did you make these?" in the future, as I get that question A LOT. So if you've been linked here, that's why :P

SO HERE'S THE TL;DR for the question "how do you make procedural space ships??":
1. I spent 2 months building a library of 3D shapes and warps to use as building blocks.
2. I research games & movies with a similar aesthetic to what I want, and pick a style of ship to build.
3. I hard-code a generic ship for that style.
4. I add randomness and detail to the generic algorithm.
5. I stretch the limits of the randomness for each part, getting to a point where it doesn't look like a ship anymore, then backing up.
6. I'm either satisfied with the algorithm and keep it, or decide that style isn't working and trash it. Either way, I might also branch it to create a new algorithm and repeat steps 2-5 for the next style!

This process has given me 3 styles that I'm happy with so far: the classic fighter style, the surreal style, and the totally-not-a-tie-fighter style.

If you're interested in knowing more than the TL;DR of how that all works, then keep reading!

Don't forget you can follow me on Twitter, where I post ship screenshots daily(-ish). In addition, I'll also link every LT tweet on these devlog threads, since I know lots of LT fans don't use Twitter XD I could also link them on the Reddit threads for each devlog... hmm...

Here are all of the ship galleries since the last devlog!

LT Ships 12.18.17
LT Ships 12.20.17
LT Ships 12.21.17
LT Ships 01.04.18


Now, onto the part that only the most hardcore fans/ nerds actually read :p



Summary
12.15.17 - 01.04.18

  • Nailed 3 ship types: classic, surreal, and tie.
  • Made & then discarded a few other ship types- vertical, radial, for example.
  • Pulled apart all ship generated algorithms into resuable parts - wings, hulls, engine, and gun all come in different algorithms than can be reusable by new ship types.
  • Started adding customization properties for all of the ship generation types.
    • Added a new UI value type for enums to allow UI selection of which ship type to generate.
    • Put a few of the randomness properties (for ex, wing length) into the UI for customization
    • Leaving alone for now; waiting for new UI scheme to be done so that I can build the ship customization tool for PAX demo.



Lots of Words About Ships

Let's break down each step from the TL;DR into more detail about how it's executed.


1. I spent 2 months building a library of 3D shapes and warps to use as building blocks

For the most part, I detailed this process in all of my previous devlogs. But let's summarize that work here just for kicks. Each item on each list links to an example of that shape, or a shape/ship with that warp applied.

Here's all of the shapes I have:
The warps & detail functions:
The code structures:
  • Shapes, for storing polys & tris
  • Joints & JointFields, for connecting Shapes

There are a couple of items exluded from each list, mainly because I haven't been using them much for the ships I've made so far. Also, the Joint system has had almost no use since starting ships. I believe it'll be useful for stations in the future, when there are many big parts attaching at joints to a main structure, but for now, using AABB-based placement for ship parts is working pretty well. An AABB basically means that I know the dimensions of the original shape that I need to attach a new shape to. This information has proved more useful than creating a Joint or a JointField for shape placement.


2. I research games & movies with a similar aesthetic to what I want, and pick a style of ship to build.

I have around 10 art books, ranging from movie & game concept art from titles like Mass Effect and Star Wars to personal art books from sci-fi artists, that Josh is lending me. I also collect images on a Pinterest board of fighter ships from places like Eve's wiki (and other games) and Behance accounts from concept artists for games like Halo.

This isn't to copy their work, but rather, to collect information on what tropes are common. Right now, since I'm building fighters specifically, I want to find what visual elements are common that tell players that this is a 'fighter space ship'. Those visual tropes are extremely important for creating procedural ships. I can't just use ANY combination of random shapes under the sun. As much as I'd personally like to create extremely surreal alien architecture, from a player's perspective, you need to be able to look at a ship and (hopefully) be able to visually classify it as armed or not- and if armed, how dangerous. So, weapons need to be recognizable, and armed things like battleships and fighters need to be recognizabley separate from cargo ships and satellites. Sampling tropes from other art is how I recreate them- and getting tropes from a LOT of different sources is how I get the inspiration to make the trope usage unique.

Within each large ship class, like 'battleship', 'cargo transport', 'corvette', and 'fighter', I then come up with sub-classes to focus on for each algorithm. I didn't do a lot of planning a head with this, but rather, I'm figuring them out as I go. These sub-classes are the 3 types I've presented so far- classic, surreal, and Tie fighter, all within the 'fighter' class. The sub-classes probably won't be a named thing from a gameplay standpoint; they're mostly helpful for coding purposes. I don't think that one algorithm can cover a whole class- it would have to be random while maintaining coherence for lots of different types of ship parts. The split between the surreal sub-class and other fighter sub-classes is the best example of this. I've tried adding surreal wings to the other sub-classes, for example, and it just makes them TOO random. It's good to have a separate classic algorithm for figthers that fill the Freelancer-like trope, without stretching the bounds of recognizability too far.



3. I hard-code a generic ship for that style.

The last thing I do before writing any code is sketch out a very generic ship for the sub-class I'm going to code. I then use that to break down what parts the ship will need and how they'll be arranged. The only exception to this so far is the surreal sub-class, which is really just an algamation of features from other classes, but with the bounds of the randomness stretched extremely far.

On the doodle, I break down each ship part into its MOST basic shapes (cubes, prisms, etc) to figure out how I'll use my library to create it. I then hard-code the shape, scale, and position for each basic component of the ship. So far, all of the ships started with just a hull and two wings. I've found the extrusion function to be extremely helpful for this step. The hull of the fighter ship, for example, is a prism with the base face extruded & contracted to make a long, pointy hull-shape. Some wings are created with extrusion also.


4. I add randomness and detail to the generic algorithm

Now that I have a static ship, it's ready to add variation. Variation is added with randomness. For example, with just the hull and wings created, I can randomize the:
- base shape
- position
- rotation
of the hull and wings.

I then add detail with my warp library. For example, I extrude random polygons for more shape variation; I add surface detail with stellation, extrusion, and greebling; I randomly add an overall warp to the whole ship so that it curves gently on an axis.

My goal going forward is to expand on this step quite a bit. I feel that the ships are lacking in detail, and now that I've nailed down the basic shapes for a few sub-classes, I want to add lots more detail. This means texturing, actually placing engines where you see engine trails, placing guns where the turrets are being added (hopefully Josh will talk about turrets in his devlog!), and other random surface details. I probably won't get all of that done for the PAX demo (probably going to have to skip texturing), but after we get back from PAX, I'm going to keep working on it!


5. I stretch the limits of the randomness for each part, getting to a point where it doesn't look like a ship anymore, then backing up.
Words.

I continue to add randomness to the ships until things 'break'. It's hard to describe- basically, I keep stretching the upper and lower limits of every random value, and adding more opportunities for random values, until the generated ships stop looking like 'ships'. At that point, I identify where the randomness went to far, and dial it back a little. This allows me to experiment with the balance between variation and recognizability, which I've been emphasizing pretty much every devlog. We want a whole universe of ships, yet the ships need to be recognizably ships. I think this method of starting with a hard-coded, boring ship, then adding randomness to stretch the limits of recognizablility, is paramount to how this process works.

I try not to rely on version control for this, but it has saved my butt a couple of times, as I made changes that I thought looked good until a few days of contemplation. For the most part, now, if I want to make DRASTIC changes to a ship algorithm, I'll go ahead and branch it, which brings us to step 6.


6. I'm either satisfied with the algorithm and keep it, or decide that style isn't working and trash it. Either way, I might also branch it to create a new algorithm and repeat steps 2-5 for the next style!
Words.

This pretty much explains itself. When I feel I've gotten to a stopping point with an algorithm, I move on to the next one! This whole process is basically just a bunch of experimenting. I gather reference, hard-code, randomize, rinse, repeat.



On the Topic of Limit Theory and Realism

I get a LOT of comments and questions about how 'realistic' the ships look, and I want to address that here. I'm probably also going to link people to this in the future when I get questions about realism. If you've been linked here, I intend not to insult you- PLEASE understand that I get comments & questions about realism daily and I just don't have the finger stamina to answer all of them.

Limit Theory prioritizes fun, gameplay, and general player experience over 'realism'. Why? Because every element of a game, including how 'realistic' a feature is, is useless at best and obstructive at worst if it doesn't contribute to how fun the game is! When that's applied to ship generation, we're often talking about what realistically could or could not fly in space or on planets VS the variety and 'coolness' of the ship aesthetics. Unfortunately, many qualifications of 'realism' seriously limit what we'd be allowed to generate, thus limiting variety and coolness.

For example, the LT stance on artificial gravity is the hand-wavy, Star Trek/ Star Wars style: even non-rotating ships could have artificial gravity. Why? Because you won't see the inside of ships, so the more FUN option is to not worry about it at all, and not limit ourselves to ships that could 'realistically' produce artificial gravity!

I need to address another big stick in these arguments: LT ship generation does not prioritize realistic flight in planet atmosphere. Why? Again, simply because that would severely limit the variety of ships we could produce. Who knows what alien or far-future technology could come up with with respect to flight, both on and off planets, so why not let our imagination go wild?

Now, I also know that for many fans, 'realism' IS what's fun. However, this definition changes allllll the time, both on the 'actual science' scale, and from player to player! Not only do we get daily comments about how 'realistic' something does or doesn't look, the comments are often people debating with each other what it even means :P This is not a bad thing. This is a sign of y'all giving a crap about the game and I love that y'all do. I just want y'all to understand how it's hard to satisfy a general consensus on what's realistic or not. We'll always upset somebody :p

Remember also that there is a kind of realism that the ship generation algorithms to satisfy, and that's player recognizability. If you've been reading the devlogs, you know that my #1 priority with ship generation is balancing regoznizability and variety. Players need to be able to look at a ship and determine whether or not it's armed, for example. That kind of readability isn't quite tied to what physics allows, but it is tied to SOME kind of player sensitibility of what a 'satellite', 'cargo ship', and 'battleship' all look like. :)

The sci-fi genre, and especially sci-fi games, exist to STRETCH and question what 'realistic' means. I think that debating what is or isn't possible is one of the most fun parts of the entire sci-fi genre. I just want ya'll to know that at the end of the day, what is or isn't the most 'realistic' option is often lower in priority than what is or isn't FUN. This has always been Josh's policy, so it's my policy too. :)




FIN

Well there you go!!! That is, as of posting, how I make the ships in LT. I hope this has been enlighting to LT fans and at least explanitory for any poor fool who got linked here because they dared to ask me how I make procedural ships XD

Gallery links once again:
LT Ships 12.18.17
LT Ships 12.20.17
LT Ships 12.21.17
LT Ships 01.04.18

Twitter link once again.

Thank you,
Lindsey Reid

P.S. I have a bunch of tutorials on 3d geometry & shader writing up on my personal blog. I'm thinking about writing a beginner 'how to make procedural geometry' tutorial, let me know if any of y'all who are programmers would find that interesting.




Link to original: viewtopic.php?f=30&t=6416
Post

Re: The Limit Theory Email/RSS News Thread

#43
If you haven't seen them yet, Josh and Lindsey made two separate threads on Thursday, detailing everything that happened at PAX South, and stating our plans for the future. Lindsey included a video of the demo, and Josh included an album of images. To anyone that visited PAX South this year, I'm glad you could make it, and hope you liked what you saw!

In case the video won't play for you, here's a link: https://www.youtube.com/watch?v=pUTZ0KBJh_E

Link to Lindsey's post
LindseyReid wrote:
Thu Jan 18, 2018 3:47 pm
Limit Theory @ PAX South 2018 Recap
Hey everyone! If you didn't know already, we spent last weekend at PAX South showing off a Limit Theory demo at a booth on the expo floor. Since many of you couldn't make it, this post will recap what was in the demo and how PAX went. There are a few goofy developer photos and a demo video for you to enjoy :)

Here's a 3-minute clip of us playing with the demo! (there's no sound on purpose :P)





What Was In the Demo

The demo had two parts:
1, a small slice of combat in the form of a dogfighter, and
2, a small demo of the ship customizer.

At first, we designed the combat to have a goal, with the player needing to dodge bullets to retain health and earn credits from kills in order to buy ship upgrades. However, after just a few hours at PAX, we decided that making the player invulnerable and removing the goals was WAY more fun. Our idea of how big the demo needed to be for PAX was overestimated; people would drop by for a couple of minutes and really just want to fly around and shoot stuff. Dying wasn't so fun, and earning credits to upgrade parts was definitely out of the scope of what a short demo needed.

So, what we ended up with was a super simple dogfighter! The player had a certain amount of friendlies and an equal amount of enemies that would constantly respawn, and there was no final goal. Players could fly around and shoot the enemies, with a couple extra fun controls: pushing down the right stick would allow the player to spawn new friendlies and enemies, and the up and down buttons on the D-Pad allowed controlling friendlies. Up would tell the friendlies to go kill whatever enemy the player was locked on to, then return to the player; down would tell friendlies to find any random target, kill it, and then return to the player. Friendlies spawned with the right stick would follow the player in sphereical formation.

In addition, the game had a small ship customizer demo. The customizer allowed you to pick from the three fighter classes I've written so far (surreal, classic, and tie-like) and customize the shape and size of the wings and hull of the ship, as well as a couple of details like the surface detailing and overall warps. The full version will allow you to do much more, but this demo was in the small scope of what people would play with during their short stop at our PAX booth. Josh also added the ability to randomize a color pattern on the ship, which was cool and well-received.



How It Went

In short: really well! We had a constant stream of visitors to the booth with almost no downtime, and virtually everybody who played enjoyed it. We ran out of the 300 business cards we printed. People really enjoyed the dogfighting, and while fewer people used the ship customizer, those that did deeply enjoyed it and said things like "I could play with this all day!"

Josh made small tweaks to the code throughout PAX, sometimes to implement or change a feature (like when we made players invulnerable), to fix a small bug (of which we had very few), or to make a change a player asked for. One of our biggest requests, for example, was to make the control axes inverted, which we're certainly going to take note of.

Players also had a TON of fun with the 'spawn' button. We had several groups of people attempt to spawn as many NPCs as they could, with the goal of crashing the game. (Un?)fortunately, the game didn't crash from excessive NPC spawning, even when we had 10,000+ ships on screen. It did, however, start to slow down the framerate... after around 2000 ships. Thousands of ships flying in various formations, hundreds-of-thousands of pulse lasers, and all of it rendered with 2x supersampling (meaning rendered @ 4K and downsampled to 1080p) -- our work on performance optimization seems to have paid off. Players frequently expressed surprise at how smooth the game ran despite having loads of stuff going on :)

Here's one group of players who spawned 4000+ ships, with Josh smiling in the back, probably at our performance:
Spoiler:      SHOW
Image

And here's Josh deep in the code, looking super exstatic about me (Lindsey) taking a coding selfie with him:
Spoiler:      SHOW
Image



Fin

Overall, PAX was a great experience for us, both in marketing and for testing the game. We made real connections with fans old and new, from backers, to people who had heard about the game a couple of years ago, to people who were learning about it for the first time. The demo was also great test of our workflow for writing gameplay code and of our performance. We found a few things to clean up with regards to architecture that Josh and Adam are now working on. We're very, very close to going full-force on gameplay code, and the PAX demo was a great milestone for that.

Here are a couple of photos of the team in front of our booth.

Josh (left), Lindsey (middle), Adam (right):
Spoiler:      SHOW
Image

Lindsey (left), Josh (middle), Adam (right):
Spoiler:      SHOW
Image


That's all, folks! I hope this has satiated your curiosity about what went on at PAX. Here's the link to the demo video again. Feel free to ask more questions about what was in the demo and such, and thank you always for your continued support.

- The LT Team



Link to Josh's post
JoshParnell wrote:
Thu Jan 18, 2018 5:37 pm
Thursday, January 18, 2018

Greets everyone and Happy New Years / Merry Christmas (a bit late)! :wave: I had, as usual, intended to post before now, but the time since my last log has been absolutely jam-packed with pumping out work for our PAX Demo. Now that it's over, I'm excited to come back and add my own thoughts on how it went, as well as update you all on the progress that's been made leading up the the release.

Of course, I recommend reading the official PAX recap posted by Lindsey earlier today for a general overview of the event.



Progress Since Last Time...

It's going to be difficult for me to remember everything. It's rather scary how quickly we're able to implement vast swaths of functionality now that we're sitting pretty with respect to our supporting engine & lower-level systems! You'll have to excuse the wall-of-bullet points and the scatterbrained parenthetical thoughts accompanying them.

  • Added working pulse laser turrets, muzzle flashes, thrusters with engine trails, explosions, and several other effects (some ported from LTC++, some brand-new)
  • Ported & improved AI architecture from LTC++; implemented escort, formation flying, dogfighting, and a few other AI actions
  • Added AI squad management & hierarchical formation (WIP -- currently the squad disbands into individual units if the leader is destroyed, although this was actually a neat feature for the demo)
  • Added ownership and asset tracking; all relevant objects now belong to an ownership hierarchy (in the demo, assets are owned by either the human player or an AI player who is controlling the actions of enemy squads)
  • Added thruster overloading ('boosting'); added AI ability to use boost (still need to generalize to all ship hardpoints to have generic 'hardpoint overclocking')
  • Added energy grid / capacitor component for 'real' energy mechanics (like in LTP); note that we had infinite energy enabled in the demo for fun (hence why the left capacitor bar never depletes when firing or boosting)
  • Added basic manufacturing component; the home base station in the demo is actually capable of manufacturing ships using a build queue and pseudo-blueprints, although we ended up not using this functionality due to reasons given by Lindsey in the recap (complex game mechanics ended up being less approachable for a play-for-a-few-minutes demo)
  • Added basic HUD with working soft and hard target locking, forward reticles ('lead indicators'), capacitor and health meters, and optional auto-targetting (turned off for most of the demo)
  • Added disposition tracking to allow dynamic inter-person / inter-faction relations (the enemies in the demo are shown as enemies not because they are in any special 'enemy' list, but because the human player and the AI player controlling them are set to have maximally-hostile standing with one other)
  • Enabled & extended a proof-of-concept paint shader on metal assets to allow per-ship/per-station coloring
  • Added dust clouds (still need work to be more subtle at high velocities -- a problem numerous people pointed out in LTC++ :oops:); improved dust flecks to be more subtle
  • Added several post-processing effects for use with alternative view modes like the ship editor / slow-time action camera
  • Added velocity-dependent radial blur (currently a bit too aggressive?)
  • Major optimizations to rendering architecture, especially with respect to supersampling -- we ran the demo at 100+ FPS with 2x supersampling on my laptop :)
  • Made everything work on gamepads + improved engine-level support for joysticks, gamepads, HOTAS, etc.
  • Partially-implemented an active deflector shield (for fun) that reflects energy weapons -- didn't finish in time for PAX, but may keep the idea of an 'active' shield mechanic on top of passive shields
  • Improved chase camera dynamics for more 'classic Freelancer' feeling (still perfecting it, but we're definitely getting somewhere!)

Indeed, quite a lot of quality functionality came about thanks to our PAX sprint! Note, in particular, how this list, in contrast to so many of my recent lists, is very much brimming with gameplay and immediately-related-to-gameplay features :clap:



Takeaways from PAX

Naturally there was a lot to be gleaned from publicly demoing a slice of the game for the first time since last year, but I think, if I had to put my finger on something concrete (other than just 'people are really going to enjoy this labor of love'), it would be the comfort with which I walked away from the experience. Comfort that the tireless, seemingly-never-ending Royal Crusade Against the Excessive Usage of Milliseconds (aka 'performance optimization') has genuinely paid off. Perhaps even more broadly, comfort that the unspeakable hours that have been put into solving FPLT ever since the fall of LTC++ have truly been worth the pain that they all-to-frequently caused.

If I were to boot up the old LTC++ and try to push it to do what our current setup did at the PAX, it would have been a spectacular display of tears and molten lava -- the former pouring from me, the latter pouring from my laptop. We are easily pushing 10x the amount of activity with extremely high, consistent framerate. We can push 100x the activity almost as easily, although there are a few choice places to which I must make small tweaks before 1000+ ship battles will be as buttery-smooth as the 100+ battles are currently. All the while, we're not only putting more entities in the world, but the amount of dynamism is starkly superior to LTC++. That demo is running full collision on all entities -- NPC ships collide with one another and with every asteroid in the scene; ramming small asteroids sends them flying off into the abyss; slowly pushing a large asteroid is even feasible with a large enough fleet. The AI is driving thousands of turrets accross hundreds of ships, each computing viable firing solutions. Friendly-fire is turned off for the demo, but projectiles still collide with everything in the scene. The collision is still slightly coarse (you will 'bounce' off of asteroids slightly sooner than expected), but this will be remedied with the help of Adam's InsaneBSPs. All those little details add up quickly into an experience that feels genuine, deep, and rich with opportunity.

All-in-all, what we have going on is exactly what LT promised to provide, and what I've thrashed my neurons for so many years to make possible: a universe alive with activity. In the demo, that activity is a bit boring, what with the AI only thinking about shooting things. But imagine the same scale of activity applied to a diversity of gameplay mechanics -- trading, exploring, mining, patrolling, constructing, researching, so on and so forth -- and you are left with the vision of a vibrant, living universe that captured the enthusiasm of so many (including myself) at the outset of LT. A single player experience in which you are far from alone :geek:

Working on the demo reinvigorated me in so many ways. Seeing even the simplest of gameplay mechanics come together to create moments of fun; having new gameplay ideas that required only an hour or two to bring to life; falling back in love with writing AI...it was all very therapeutic :geek:

The icing on the cake, of course, was player response. As mentioned by Lindsey in the PAX recap, we had numerous players express their amazement at the sheer amount of 'stuff' going on (especially those who spawned hundreds/thousands of AI escorts, expecting a crash but instead finding only a very gradual FPS degradation). We received plenty of compliments about the beautiful visual style, the smooth and responsive feeling of it all, the joy of being able to issue orders to a fleet, the quality of the procedural assets (Lindsey's ship generator amazed more than a few people, especially once she had made clear that the ships were procedural down to the very vertex, not simply lego-style recombinations of prefab assets). Even with such a small taste of the full Limit Theory experience, people responded really positively. Yet again, it brings me comfort and excitement -- for if just a tiny morsel of the cake is enjoyable, surely the full experience that awaits you all at launch will satisfy :)



Going Forward

The sprint to PAX functionality gave us a chance to see how our various systems and architectures performed not just with respect to runtime, but with respect to developer time -- the second half of FPLT is, after all, making this raw engine power easy to harness and sculpt into something that matters. For the most part, I'd say we succeeded there as well. However, as was to be expected, pushing content into the demo in such a short time gave us critical insights into where things are too complex or less-than-intuitive.

Adam and I are going to spend the rest of the month doing a sort of PAX-postmortem, taking some time while we have it to deal with those hitches that affected our PAX demo development efficiency (and will, by extension, impact our dev efficiency going forward toward release.)

On my side, perhaps the biggest challenge remaining is moving our PAX demo into mod format. As you may recall from my last log, the mod format was well-underway at the time. As it stands today, it's more-or-less complete, but yet again, moving over representative gameplay snippets like some of those developed for the demo will reveal the true efficacy of a system that looks fantastic on paper.

I'm really looking forward to the next few weeks, as I think it's going to continue to be a lot of fun writing new gameplay, listening closely to see what the code is telling us about our architecture, and iterating until we're doing bigger and bigger things with less and less effort. The PAX demo marked the true start of the ramp-up into gameplay code. The rest of the month will be an outward spiral of ramp-up and ramping-up the ramp-up (you know, d2G/dt2, where G(t) is gameplay :geek:).

Thanks for listening :wave:



I put together a small album of some of the screenshots I took during PAX (and a few after). Mostly of people doing absurd things :V Please note that the radial blur & horizontal screen effects were just a little PAX polish for making the camera modes look distinct. LT does not have the outer portion of the screen blurred out most of the time :ghost:

Limit Theory Devlog Gallery ~ January 18, 2018

Image

Image

Image
Have a question? Send me a PM! || I have a Patreon page up for REKT now! || People talking in IRC over the past two hours: Image
Image
Image
Post

Re: The Limit Theory Email/RSS News Thread

#44
Wednesday, February 15th, 2018

Allllright. It's been a little while since I've rambled on the forum so I figure it's time for an update. Plus, with Lindsey down for maintenance and Josh defragging the system someone's gotta keep y'all in the loop, right?


It looks like PAX was a little more draining than we expected. I'm certainly not complaining, because we had a great time jamming on the demo and having so many excited people hanging out at the booth all weekend. Committing to writing only gameplay code for a week and getting to see just how well all the moving parts are operating together has been a massive kick in the pants in terms of motivation and excitement. We were able to absolutely slam through features and have everything Just Work.

That said, I think everyone is finishing up a little recharge. December and January have been hectic. Between travelling for Christmas, jamming for PAX, presenting at PAX, the Global Game Jam, and moving we're all taking it a little bit easy right now.

What have I been up to since you last heard from me, you ask? Well, let's see..

UI Iteration
Since the original implementation of the dev UI I've made some really nice simplifications and improvements. In fact, Josh decided to move forward with the dev UI and the final gameplay UI. That meant I had to do some more hammering to ensure it was up to snuff. I massively simplified how focus and dragging are handled. I made sure that the UI does sane things when widgets are removed at runtime. I added tons of polish: keyboard and gamepad navigation, ScrollViews always keep the current selection in view (even if you were to nest multiple ScrollViews inside each other), fixed the last few subtle 'off-by-a-pixel' type issues, and started pulling over Josh's shader-based UI drawing code. That's right, just because it's programmer art right now doesn't mean it's staying that way. It's super easy to swap out the drawing code with proper prettified versions.


General Infrastructure
I tend to alternate between big tasks and little tasks to keep my mind fresh so I frequently bounce around making little things better. We now have proper signal handling in the engine. In unexpected/unsupported situations the engine simply aborts. Previously this meant the game just printed a message and closed. If you had a debugger attached you'd have a chance to walk up the call stack and determine what happened, but sometimes you don't have the debugger running (or never, if you're a masochist like Josh). Now we can easily register handlers from any system that will run any time the game crashes. A particular pain point was Lua. Being a dynamic language it's super easy to write silly bugs that will cause a crash (like type the wrong variable name when calling into the engine and accidently passing a null pointer instead of, say, the mesh you want to render). Now we're able to automatically print the entire Lua stack when something goes wrong which instantly cut our bug fixing turn around time by an order of magnitude. 9 out of 10 times we know exactly which line caused the problem without having to think about it.

I also went through the engine to standardize and organize some of it a bit better. We had various APIs with unintuitive names to avoid name collisions with other common libraries. So things like StrUtil, MemUtil, and LibMath became PhxString, PhxMemory, and PhxMath. I also ripped out some templates in favor of good old function overloading.

Before we had a solid Lua foundation we used a second C++ project called scratch where we did explorations and wrote test applications that stressed specific aspects of the engine. In particular, that's where my (fairly large) BSP testing application was. Now that we have robust Lua tooling we write our tests in Lua based applications that actually live in the LT project and we're able to easily choose which application to launch. That scratch project has been sitting there unused and collecting dust so I rewrote the BSP tests in Lua and nuke scatch entirely.

I also added some niceties to our Windows build environment. I massively trimmed the output so it doesn't spam the console with things we don't care about, added a brightly colored message for build success/failure, profiled build times in an attempt to cut them down (turns out, vcvarsall.bat takes longer than compiling and optimizing over 16,000 significant lines of code, thanks Microsoft!), and output colorized warnings and errors in realtime. I think I finally learned my lesson this time: Command Prompt / Batch files suck. I'm done with them for good this time. I'm going to pick a new language for tools.


Input
Here's where I've been spending most of my time recently and where I think it would be fun to dive in a little.

Input is something I've wanted to sit down and really solve for quite a while now. There's a lot of subtlety surrounding input and finding a way to handle the complexity while maintaining a level of elegance is a worthy challenge. In my last game I dealt with input as 2 separate type: Buttons and Axes. Buttons were a binary thing, either pressed or not. Axes were an analog thing, a value in the range [-1.0, 1.0]. The biggest issue here is that you have to decide in advance what is a button and what is an axis and you have to stick to that. You can treat the same input as a button in some places and an axis in others. For example, the left analog stick on a controller wants to be treated like an axis when you're moving your character, but in the main menu you want to translated it to button presses for Up, Down, Left, Right. My way around it at point was to have wrappers that converted between input types in the actual bindings. The types ended up looking something like this:

Spoiler:      SHOW

Code: Select all

interface IButton
{
  bool IsPressed  { get; }
  bool IsHeld     { get; }
  bool IsReleased { get; }

  void Update();
}

interface IAxis
{
  float Value      { get; }
  float Deadzone   { get; }
  bool  IsInverted { get; }

  void Update();
}

class ButtonToAxis : IAxis
{
  private IButton button;
  // ...
}

class AxisToButton : IButton
{
  private IAxis axis;
  // ...
}

Obviously this was back when I was a lot OOP-ier (read: wronger). There are lots of things I don't like about this now: bindings are doing doing actual input processing instead of just mapping values to actions, it's a polling interface, it has interfaces, inheritance, and virtual functions...

The polling style is the biggest issue. You end up checking every possible action every frame. And what happens when a button is pressed twice in the same frame (unlikely, but possible with low framerate when typing text)? How would you record and replay input to implement a replay feature for gameplay or debugging purposes? If you're serializing bindings in order to save them you now have to store information to about this ButtonToAxis/AxisToButton business. It doesn't feel like the right solution.

What if, instead of asking if a button is pressed, we stick events in a queue for every change in input. Then gameplay loops over the input queue and runs the necessary logic for each button press or axis change (I stole this idea from the Quake engine). That feels like a step in the right direction.

So my first attempt at improving our input handling was to implement an InputStream API. Here, everytime the engine detected an input change it stuck an InputEvent into the stream. From the gameplay side we'd loop over all events at the top level of the game loop and call OnInput on everything that was currently consuming input. LuaJIT was not a big fan of this loop. And it didn't solve the button vs axis issue. But, we did get gamepad support as part of this change. And we no longer missed multiple presses in the same frame or a press and a release in a single frame. However we still had to treat buttons and axes differently and we were shoving extra information into the events like the mouse position/scroll and the current modifier keys. Still, that's a net win and this is the system we used at PAX.

Then, Josh gave me 2 small nudges to see the full solution: 1) why are we sending input events to everyone instead of the just the place that uses it, and 2) what if everything was both a button and an axis? All input, regardless of its source starts off as an analog value. Then we take that analog value and interpret it as a button also. Let's say when the value is greater than 0.5 it's pressed and less than 0.4 it's released. We store both and let gameplay decide which representation it wants.

At this point, we still needed a proper solution for remappable bindings, support for more than just the Xbox 360 controller we used a PAX as well, HOTAS support, and haptic feedback. However, we still wanted to keep the ability to read input without setting up bindings and to query input directly without using the event loop. When we're writing a test application or we're prototyping gameplay we don't want to have to think about bindings and events we just want to do the braindead thing and ask if the space bar is pressed. We needed to keep all the existing fucntionality of the 'inferior' input APIs (at this point we're up to 3 different input APIs)

Thus, for the past few days I've been rewriting the input system to incorporate all these ideas into the One True Input API that deprecates all the others, solves every problem, and ends world hunger. I think I've accomplished the goal (well, am currently accomplishing. There's a little more to go). I ended up split the input system into 2 parts: a low level API (Input) and a high level API (InputBindings).

The low level API is as simple as possible. It reads input from the system, keeps track of everything as both buttons and axes, and pushes events into a queue. Because it stores a copy of the current input we're able to do the direct queries: Input.GetButtonPressed(Button.Space) but it also stores the number of button transitions during the last update so it won't miss double presses or press and release in the same frame. It leverages SDLs ability to virtually map all controllers to a standard Xbox 360-like layout so just about any controller you plug in will Just Work. It ensures press and release events are perfectly paired (even if your controller is remapped while holding a button, much to Josh's annoyance) This API is extremely robust and basically complete. Everything LT related has been migrated over. It needs slight improvements to the way modifiers are handled, haptic feedback support, and to be tested with HOTAS / unmapped controllers.

Spoiler:      SHOW
Image

The high level API is where bindings and the callback magic happen. Multiple inputs from any device can be bound to an action. Actions 'eat' the input so only the most recently added action will be triggered which allows us to push sets of actions onto a stack. Callbacks for specific actions (e.g. OnPressed or OnValueChanged) can be registered for from Lua and code will be called directly rather than have to query or process events. The high level API is meant to mirror the low level API as much as possible (making it painless to move from the low level API to the high level one) so it supports both direct query and an event queue in addition to the callback mechanism. The API is mostly implemented but we need to flesh out the callback mechanism, the syntax for creating bindings/registering callbacks, and explicit device assignment.

Honestly, I'm quite proud of how this is turning out. I feel like I've 'solved' input and will be able to use this approach for future projects. At the very least it's nice to have reached a point where we have one simple, unifed way of dealing with input.


Misc
Other stuff I've been up to:

  • Integrated FMOD for audio support.
  • Attached very quiet cantina music to asteroids and watched Lindsey go slightly insane trying to figure out why she has Star Wars stuck in her brain.
  • Sniped revisions 1000 and 1024 from Josh and made really boring commits.
  • Played around with tooling for merging / viewing / debugging mods.
  • Moved in with my girlfriend.
  • Got a puppy!
  • Gave said puppy a trial run as head of the Office Happiness Enhancement Department
    (she'll be back for a second round of trials in March).


Phew. I've glossed over so much detail (like the ridiculously awesome procedural sound that's on the horizon) and this is still a wall of text. How about a picture of Tess to make up for it?

Spoiler:      SHOW
Image



Link to original: viewtopic.php?f=30&t=6449
Post

Re: The Limit Theory Email/RSS News Thread

#45
Friday, March 9, 2018

Happy Friday everyone!! As usual, it's been too long since my last wall of text, but boy have I been busy since then...and I'm very much hyped to show you all what I've been up to :) I won't belabor this intro (EDIT: guess what: I failed :V), since I know you're skipping straight to the screenshots (yes, you, I know what you came here for :ghost:).



Last time, I spoke of moving forward with the fun stuff -- that juicy gameplay -- and mentioned that 'perhaps the biggest challenge remaining is moving our PAX demo into mod format.' Yes folks, after 5 years of this, I've actually gotten moderately decent at predicting where I will stumble. And stumble I did...over and over and over again, until I was dreaming of mod files with teeth chasing me down. I tried looking at the problem (of writing in mod format) every which way: upside, downside, yonderside, and they all upset me in their own special little way.

So I said "frak it," and decided to drop modding support...

...in literally zero of the infinitely-many parallel universes of the quantum multiverse, including our own. No one believed that, right?? I'll lay down and die before I give up on making LT the 'OpenSpaceSim' we all want! But in all seriousness, I did get tired of trying to solve the problem of making modular additions to it easy to write with some text format. Frankly, I'm just tired of text. At least, I'm tired of text that represents code. It bothers me every day. I spend most of my life looking at Vim. Yes, I have a deep affection for Vim; yes, I have a tricked-out .vimrc and a lovely custom color scheme crafted to my tastes; but at the end of the day, it's all rather uninspiring. There's a pretty severe disconnect between the Josh that wants to teach the computer to build a beautiful, awe-inspiring universe and the Josh that sits in front of a slate-blue wall of code 8-to-16 hours a day :monkey:

The harder I pushed to write the game in a modular format, and the more I tried to nail down that modular format, the more I came to realize a devastating (at the time) truth: seeing the big picture and being able to build the big picture from small, incremental changes are two goals that are fundamentally at odds with one another when one's way of seeing the big picture is the same as one's way of changing it. I guess that's a bit obvious when put abstractly, but at the concrete level of writing game code, it felt like a disheartening revelation. But all was not lost. You've glanced at screenshots already, you know what's coming, so let's quit beating around the bush.

I spent two weeks prototyping the viability of a dedicated editor for Limit Theory. At the end of those two weeks, I had something that worked, something that enabled me to see the big picture all at once, while being able to translate my changes behind-the-scenes into the tiny, incremental changes necessary for good modularity. Those two weeks changed the whole ballgame. You may or may not remember that I showed a screenshot or two of a 'failed experiment' in building such a tool in a KS update a while back. So what changed? Well, the viability of building such a tool shifted dramatically when good ole' Adam mentioned to me a library specifically made for getting functional, practical GUI tools up-and-running with minimal effort -- Dear ImGui. Previously, I had used Qt. Qt is great; it's a behemoth of industrial-strength functionality. But the time cost of building things is orders of magnitude higher. Two weeks using Qt left me feeling that I was going to spend more time on tools than I would save on getting LT out the door. Two weeks using ImGui left me feeling that I had an infinitely-more-powerful workflow that would make the road to release so much more enjoyable, shorter, and paved with less frustration. Two weeks with ImGui made me a happy Josh :)

So then, let's have a look at the new, blossoming development tool that has allowed me to sleep soundly at night!



We'll hop right in to the main editor view:

Image

There's a lot going on in this screenshot, and it's more cluttered than the normal workspace would be since I'm trying to show off a lot of the functionality all at once. In the center area we're looking at the definition of a 'Ship' entity. We can see everything all at once -- the components that make it up, the data and functions imparted by those components, along with Ship-specific data. It's a 'big-picture' view (from the code standpoint) of what a Ship is in LT (note: it's not complete, as I'm not finished building the 'Core' LT mod file yet!)

Something really important to note here is that I've actually got a small test mod loaded in the editor, but that test mod depends on 'Core' (Vanilla LT). Since it depends on Core, what I see in the editor is the result of Vanilla LT plus my changes to it. Big picture view!!

But! Glance over at the center-right panel, and you will see the actual mod file that I'm producing. It's a Lua file, automatically-generated by the editor, that tells the engine all the things that my mod adds/changes/removes. If you take a look, you will see that it's essentially just a list of tiny, incremental changes. Extremely granular, and just about as ideal for seamlessly merging the changes from lots and lots of different mods as one can get. Therein lies the real beauty of being free from a pure-text/code workflow: the big picture that we see is completely decoupled from the description of building that big picture. We get the best of both worlds: the ability to understand everything in the game at once, the ability to modify any piece of it, and the generated mod file that describes those modifications as compactly as possible, thus maximizing modularity.

(If you're wondering, this mod was a test of cross-platform compatibility, making sure that I could load/save a mod, swap between Windows & Linux, and keep working without issue, since I use the latter while Adam & Lindsey use the former. It was a successful test, fyi :) )



From here on out I'm just going to show some of the functionality of the editor and talk a bit about how it works, why it makes life easy, and so on. Using this tool, we can quickly create new entities, components, events, functions, fields, etc. There is never a question of what file it should go in, how we should structure the syntax, etc. "Minimal degrees-of-freedom" is a major design goal I have for our workflow. Here I'm adding a new function to the Inventory component, to make it easy to transfer items between two entities:

Image

The code view is rough and there's no syntax highlighting or autocomplete or anything like that -- but frankly, the whole point here is to get away from unstructured code as much as possible. As time allows, I'll improve things like that, but my core focus is on making it as fast and intuitive as possible to build new game content, while ensuring that understanding existing game content is always tractable. Note that I'm doing this for me more than anything, because I need a workflow like this to keep a handle on the beastly featureset of LT. It has overwhelmed me once before; but not this time -- this time I have all the power in the world to keep it simple and manageable :) :squirrel:

After I've added my new function, it shows up in the Inventory component or as part of the 'All Component Functions' listing in any entity that has an Inventory:

Image

On the right I've got a panel showing me the new function. The central area of the editor can have as many 'vertical split' panels as your screen has room for, so it's very easy to look at, for example, an entity and a specific component within that entity side-by-side, or compare two entities, etc.



Ok! Let's do a quick little demo of extending the game :) (note: in reality, the contents of this mod will be part of Vanilla LT, but this is just for demonstration!)

Creating a new mod that extends Core:

Image

A few minutes later, I've built a new component to hold the data & logic governing a 'population' (like on a planetary colony, or perhaps even a large station). I've added it to the 'Colony' entity, and now every Colony in the game will have this data and run the logic that I made! On the right, you can see the generated mod file:

Image

Notice how everything can (and should) have a description, and how these descriptions pop up as tooltips when you mouse over just about anything in the editor. This is all part of keeping the 'big picture' as clear and understandable as possible! If/when you open Vanilla LT in this editor, exploring and understanding how it works will be a lot more fun than just reading some documentation :nerd: (of course, I'll provide that too).

I can save my mod and load it again later when I fire up the editor. I'm greeted with this, reminding me of how awesome my little mod is going to be :ghost:

Image

And there you have it! I've now got a single Lua file that changes Vanilla LT to (presumably) make it even better! Ship it! :clap:



I mentioned earlier being pretty disheartened at the lack of an 'inspiring' workflow. We are now in a position to obliterate that issue! On top of the ImGui Lua bindings, I've also written some OpenGL Lua bindings that, among other things, let us put some 'game' back into 'game development'! Here I'm perusing LT screenshots in the file browser:

Image

Goodness, that's refreshing to see some beauty inside the development tool :D Here we have some really basic OpenGL integration (not exactly beautiful, but the point is that we can use this same tech to visualize LT-related things in-editor):

Image

And finally, a fun addition that I wrote last night (it took all of 30 minutes to do since this is Lua, after which I spent hours playing with it in the editor :geek:): a real-time shader editor for ultra-fast prototyping...and maybe just maybe for boosting a certain graphics monkey's spirits if he's having a rough day :monkey:

Image

These are just tiny demonstrations of the power that we now have on-hand. If and when we need to see something, to simulate something, to prototype something, to measure something, we can do it in an awesome way without sinking huge chunks of time into scratch/sandbox applications (and without inducing a mental context switch to that other application!)



In closing, I'll summarize a few points just to make clear where we stand with development, the editor, and the productivity level of my past few weeks! Here's what we've got now:
  • A finished mod format that's functional, granular, and generally awesome :thumbup:
  • Support in the mod format for whatever 'categories' of game content we want to add in the future, for example, Item Types, AI Behaviors, Procedural Generators, Settings, and so on (if you look carefully you can see that the mod format handles these all in the same, unified way)
  • Ability to create new mods (that can have any number of dependencies, or none -- Core is built the same as any other mod, it just doesn't depend on anything!), load & save them, automatic backups, visual highlights on changes made by the active mod, arbitrarily-deep undo & redo (which is actually tightly-related to the mod format), ...and so much more
  • All of the above written in Lua, making it super-fast for me to add functionality; ImGui functions are bound & injected into Lua automatically
  • In-editor OpenGL integration and Lua bindings for OpenGL, making it possible for us to create development visualization & simulation tools limited only by our imaginations (procedural algorithm rapid iteration, material editors, historical simulation sandboxes, dynamic economy simulations, you name it! As a bonus, since it's in Lua, we can run pieces of the game code directly in the editor, no extra work required!)
  • An engine-side loader & compiler that takes in an 'active mod list' and compiles the merged data down to code that interfaces with all of our existing tech (ECS/CTypes, engine-side functions, Lua utility libraries, etc.)
  • An inspiring workflow!!
  • A 'Core' mod representing Vanilla LT that is already well-underway
  • A dramatically-nicer workflow and a happier Josh :)

And in case this wasn't implied, LT will ship with this editor for your use in modding.

---

Wow. That was quite a mouthful. If you made it this far, thanks for sticking with me! It's always interesting to look back on the devlog process; it really gives me insight into how I'm actually 'feeling' about development, which can sometimes be difficult for me to process, even when...I'm me (which is most of the time :V). To see such an explosion of excited JoshTalk in a relatively small devlog session, and to see it come so naturally and enthusiastically...it boosts my spirits even more. The road to release has thrown us more than a few hardballs, but every now-and-again, we catch a break. I couldn't be more thankful for this one.

I anticipate the next few weeks being focused on continued development of 'Core.ltm.lua,' getting all those lovely entities and components fleshed out, and continuing to leverage the editor at every opportunity to make life easier and development more enjoyable. The next big milestone will be when I can hit Ctrl-R in the editor and have the game launch moments later, running off of compiled mods, thus completing the feedback loop of fast development iteration.

Keep your lasers hot and your missiles locked my good fellow pilots; it's never been a better time to be building universes :D :squirrel:

---

You've already seen all of the screenshots, but here's the link to a gallery of this log's images: Limit Theory Devlog Gallery ~ March 9, 2018




Link to original: viewtopic.php?f=30&t=6459
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford

Online Now

Users browsing this forum: No registered users and 5 guests

cron