Return to “Announcements”

Post

Re: The Limit Theory Email/RSS News Thread

#31
I've decided to include a small post by AdamByrd (one of the two coders Josh hired as help) as a temporary stand-in while we wait for an update from Josh himself. :)
AdamByrd wrote:
Tue Aug 15, 2017 9:59 am
Guys! Gals! Hey. I'm just popping on to check in with a super ultra tiny mini update.

Things are starting to move fast. Lil'T is alive and kicking. And producing pixels. Actual pixels! (Seriously, I've spent so much time staring at text books and data dumps these last three months I almost forgot we're making a game. With nebulas and rocks and space pirates.) I've been working to nail down how we're going to be writing actual gameplay code. That means fleshing out how Lua talks to C to keep heavy bits and allocations in fast code land, how to make that Lua feel 'normal' and not like C grafted onto Lua (metatables are really nice!), making sure LuaJIT actually jits to assembly, iterating on the engine API to remove friction, and building a prototype solar system. I'm insanely excited about the progress we've been making. All the mountainous unknowns are now small piles of exploded rubble. We're moving faster by the day.

The GIF floodgates are opening soon, my friends.

What a time to be alive.


Link to original: viewtopic.php?f=30&p=153664#p153643
Image
Have a question? Send me a PM! || People talking in IRC over the past two hours: Image
Post

Re: The Limit Theory Email/RSS News Thread

#32
Friday, August 18, 2017

Hey all!

As usual, I apologize that these logs are doing a bit of temporal stretching lately -- honestly we're just really, really busy working on LT and it's tough to pull myself away these days, so consider it a good thing! When we last spoke, several major advances had set us up for fruitful weeks to come. I'm pleased to report that they were, indeed, very fruitful.

At the moment, I can summarize our work by saying: we're dangerously close to shifting the majority of our efforts to sprinting on LT gameplay code in Lua. And we're doing everything we can to close the gap.

Seriously, the 'road to (performant) gameplay' has been way longer than anyone wanted. But the end is in sight :)

---

Josh

  • Finished implementing zero-cost Lua wrappers for engine functionality (mentioned in last devlog; means we're getting superb performance when interacting with the LT Core)
  • Implemented enough of the foundational Lua code that we can all work on LT in Lua at the same time
  • Continued careful analysis of LuaJIT debug output; continued to ensure 100% JITing to assembly; continued to learn more about the trace compiler and how to beat it into submission :ghost:
  • Hooked up our ECS to Lua + wrote the convenience wrappers that allow it to be used really easily (note: we now call it the 'CType' system instead of 'ECS', reason being that it is actually more general and flexible than an entity component system, although it can and does serve as one -- I'll explain more below!)
  • Ported Lua deferred renderer; modified to take advantage of all high-performance constructs
  • Made major improvements to our logarithmic z-buffer code while doing the above (this is the code that allows us to render objects at massive scales & distances without issue)
  • Implemented a work-in-progress event system in Lua for use in gameplay code
  • Implemented Lua wrappers for efficiently dealing with SDFs and OpenVDB (the library mentioned in the last log that we now use for extracting meshes from distance fields)
  • Implemented CSG functions on SDFs that are roughly 10x faster than OpenVDB's defaults, allowing complex, detailed meshes to be built very quickly (read: better procedural mesh algorithms coming soon!)
  • Wrote a simple pathtracer for fun since graphics Josh has been repressed :monkey: Not totally irrelevant to LT, though, as we could perhaps use it for nice renders of procedural assets or for having a 'ground truth' reference for our physically-based rendering code
  • Built several Lua 'template' applications that allow us to easily various, specific pieces of functionality really quickly (just like the old LTSL sandboxes!)
  • Loads of other things that I'm forgetting

Adam

  • Started working in the Lua side of the LT codebase!
  • Put in the time to learn about all of LuaJIT's lovely 'idiosyncrasies'; Adam is now 100% capable of helping me deal with the technical side of LJ
  • Used LuaJIT magic to implement the first parametric type in our CType system -- the arraylist! This means that not only can we define and use new, native datatypes in Lua, but we can also have what amounts to native std::vector<T> and so forth of them. I know I've beat this horse to the ground, but seriously, the power we now have to create and use types that are as fast as compiled code simply can't be overstated! Memory management is by far the weakest spot in Lua's performance, and we are almost entirely side-stepping it.
  • Hooked up BSP code to Lua and used it to implement picking (being able to click on an object in the world and do something in response -- a seemingly-trivial task that is not at all trivial for large scenes and complex meshes, since it requires casting a ray through acceleration structures)
  • Got our first star system set up and working using CTypes and all the new high-perf Lua stuff!
  • Made a lot of improvements to our geometry code (BSPs, triangles, rays, etc.)
  • Implemented really fast BSP-Sphere intersection tests
  • Loads of other things that I'm forgetting (again)

Sean

Sadly, last Friday was Sean's final day here as an intern, since he's now headed off to college to study CS and become an unstoppable force of programming. We'll miss him for sure...his time here seemed way too short. Nonetheless, he was able to accomplish an impressive amount in the seemingly-little time that he had.

In his final act of glory, Sean finished implementing a fully-generic, hierarchical, fleet-based AI maneuvering system, capable of representing fleets with any number of sub-fleets, and intelligently directing their motion. To demonstrate it, he showed us a group of several fleets flying together in a V formation, with each fleet comprised of squadrons in a V formation, each of which was comprised of fighters in a V formation...you get the idea! All of the math required to make it happen is rather difficult due to this arbitrary nesting requiring scale-invariance. Sean got it all hammered out and now we have proper, hierarchical fleets. I'm really excited to see this in-game :D I don't have a gif on-hand, but I'll be sure to show off his work as soon as we attach it to the main LT Lua code.

---

High-Performance, High-Convenience: A Peak at Coding in LT

As you all know, Adam and I have worked relentlessly over the past months to devise ways to achieve both extreme performance capable of handling the LT simulation logic, as well as the convenience of being able to write said logic in a language that won't rob us of every last drop of life. I'd like to show just a snippet of code demonstrating what our native type system ('CType' -- the final version of what we used to call the ECS) can do, how it looks, etc. So, have a slice of toy code! (Since it's harder to read without syntax highlighting, I just screenshotted some code in-editor)

Image

This is a massively-simplified battle simulator, intended only to showcase the simplicity of gameplay code. Nonetheless, it demonstrates some important features / fruits of our labor. Note: I cranked this out really quickly and didn't actually test it, there may be typos or slight mistakes :whoops: )

Notable Take-Aways:
  • Native memory types can be defined on-the-fly in Lua; behind the scenes they are 100% as compact as the equivalent type would be in statically-compiled C; accessing the memory is extremely fast due to fancy, custom memory-pool allocation magic done automatically by the engine. All of this complexity is hidden from the user, so code remains very easy to read and write.
  • Types can be nested in other types, enabling 'component-based' design, as demonstrated with Health and Weapon. Of course, since this is a full-blown type system rather than an ECS alone, we can do things like have multiple health components, which ends up being useful sometimes!
  • 'Member functions' can be attached to these types and called in the idiomatic Lua manner; convenience + performance = winning!
  • Generic functions like 'Weapon:damage' can be written without concern for the type of the target object and still 'just work,' despite the fact that we're using direct field access instead of table lookups like most dynamic languages (if you're a programmer and understand what I mean, that should blow your mind a bit :) )
  • (Here's the even-more-mind-blowing magic) Functions that can operate on different types of objects (like Weapon:damage, which could be used to damage stations, hardpoints, etc) will be traced by LuaJIT and end up running as fast as equivalent statically-compiled constructs (in C this would require a switch statement or function pointer, in C++ this would require a template or virtual tables; here it requires nothing extra!!); this is a HUGE win for both performance and simplicity!
  • Although I didn't show it above, types can be built in a piece-meal fashion rather than defined all at once. In particular, mods have access to type definitions and can inject their own fields and methods as required to support extra, mod-related data and functionality (I showed something like this in a devlog a while back). As always, giving modders the ability to add just about anything conceivable to vanilla LT is really important to me!

---

Well, once again I've completely and utterly failed to be brief :ghost: :ghost: It's hard to be brief when there's so much happening and so much about which to be enthused. Speaking of being enthused. I must leave you all now to return to the code :)

:wave:



Hello, everyone! Talvieno here with a non-technical summary. There hasn't been much demand for one for a while, but this devlog seems to be worthy of it. Without further ado, let's get down to business! There are a couple "big things" to take away from this update - namely, Sean is no longer with us, and we're almost ready to start focusing entirely on gameplay. As to Josh's bullet lists, I'll break down what those mean in the spoilers below.

Josh
(tl;dr: made things much faster and more performant)
Spoiler:      SHOW
  • Implemented stuff for better performance
  • Made it easier to write code while the game is running (I think)
  • Implemented more stuff for better performance
  • Hooked up pieces of code we've been working on for a while so we can actually use them, but then called it CType (instead of ECS) because that sounds cooler and makes more sense
  • Some stuff that printed stuff to screen that used to be in Lua is now in the C code core, so it's faster
  • Made major improvements to code that let us draw things that are far away (or very large)
  • Worked some on the foundation for gameplay code
  • Implemented stuff for better performance (Yes, again)
  • Implemented stuff for way better performance (No, really)
  • Got sidetracked and made some really shiny images with what seems to be a homebrewed renderer (which is particularly impressive, if not entirely LT-relevant, but it was done in Josh's free time)
  • Built some programs to let us test small pieces of LT Lua code quickly
  • Lots more where that came from

Adam
(tl;dr: Lots of coding wizardry)
Spoiler:      SHOW
  • Started working on gameplay code and gameplay-related things!
  • Fully educated himself on Lua and LuaJIT, making him roughly on-par with Josh's level of accumulated LuaJIT knowledge
  • Long complicated bulletpoint that essentially means Adam is a wizard and is now quite near god status. Basically he implemented a lot of techy workaround stuff that overcomes Lua's inherent flaws. Neato, right?
  • More wizardry. Like Josh said, Adam implemented stuff that lets you click "on" an in-world object. Trickier than it sounds. It's taken care of now. It seems this may be more robust than the way Josh did it before, too.
  • Created a star system with the new LT code he and Josh have been working on. I basically copied this from what Josh said, it's pretty clear already.
  • Implemented more stuff for better performance (yes, Adam is guilty of this generic task description too)
  • Implemented some really performant collision-related tests, meaning we can check for collisions even faster. (Collisions are actually really hard for computers to detect. Our brains are much better suited to it.)
  • Loads of other stuff that Josh is forgetting.
Okay, now we're to where Josh is telling us that Sean is leaving. :( We'll miss him dearly, and hope to see him around sometime in the future. In his final act of glory, he [technical gibberish that basically means he made ships in fleets fly around in an actually smart way, instead of all flying around haphazardly like in the LT Prototype]. It'll look pretty, believe me.


As to all the last part (the image, the code, the final bulletpoint)... that can be summarized by saying Josh is proudly showing off the fruits of his labor, which unfortunately may not mean much to anyone that isn't a well-established programmer that can at least code on an intermediate level. The main things to take away from it is that it's fast, and flexible, meaning it's the perfect tool for modders. I'm actually understating it here - this is some top-tier wizardry that Josh and Adam have written up. If you're used to games running considerably more slowly when you start adding mods, that shouldn't be the case with LT. For those interested in writing their own mods, all this also means that you'll have a ridiculous amount of control over the functionality you can add to the game.

And, that's it! Josh failed epically at being brief (does he ever not? We love him for it), but I've hopefully condensed it down to something a bit more pleasant to read for the non-technically inclined. :) I'll answer any questions as necessary, and thanks for reading!



Link to original: viewtopic.php?f=30&t=6271
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: The Limit Theory Email/RSS News Thread

#33
Thursday, August 31, 2017

Really happy to report that it's been a tremendously productive two weeks! In the last log I spoke extensively about our various mechanisms for getting bleeding-edge performance out of a magic blend of LuaJIT and C. Since then, we've been building more and more 'real stuff' using those systems, working our way to release-quality code! The general theme as of late has been "wow, it just works!" Of course, the longer version is "wow, after 1000s of hours invested in technobabble and aggressive banging-head-on-desk followed by burning the codebase and restarting, things finally, at long last, just work!" :ghost:

---

(I'm not going to separate by Josh & Adam this time since we were both working all over the place.)

Basic Game Objects & Components: Working!

Last week was the first time that we were actually able to build and use some real components and game objects purely out of our CTypes system (see last devlog if you don't know what that means). It worked beautifully. Right now we've only got System, Asteroid, Ship, and Nebula objects -- we're scaling up slowly as we nail down what we want the gameplay code to look like / how we want to be able to manipulate said objects in Lua. We've also got some basic components -- Motion and Transform, in particular. The former handles kinematics, the latter keeps track of position/orientation, etc. These are your bread-and-butter components for a component-based game engine (along with something like Drawable/Renderable, Inventory, etc.), so naturally we want to test the most ubiquitous components first.

The real stress test of our CTypes system came when we packed thousands of asteroids into a system, doing a rather naive update loop on them, and performing full rigid body dynamics on them (meaning I stuck a motion component in the asteroid definition and allowed every single asteroid to have accurate linear and angular dynamics -- something that we're probably not even going to do for release, because I still contend that dynamic asteroids don't add a whole lot of value..but hey, mod away). The speed was unreal. The bottleneck was, by a huge margin, the rendering (no LOD models yet). The actual update logic is so fast that, if I turn off rendering, I can get up to about 30,000 asteroids before I drop below 60 FPS on my machine. Again, that's a full rigid body dynamics step happening on 30,000 moving, spinning asteroids @ 60 FPS. And that is using very naive update logic, no cleverness about putting objects that aren't moving to sleep, no threading, etc. Adding in cleverness is only going to push that number even higher :shock: Anyway, I guess I can stop bragging about our performance now :oops: It's just that...I spent SO long battling FPLT -- and sometimes seriously doubting that I'd be able to give LT that 'epic' scale that I so badly wanted -- that...surely I deserve a little leeway... :monkey: :angel:

Dynamics: Fast, Accurate

I mentioned the rigid body sim above, but it's worth mentioning it as a stand-alone point. To be perfectly honest, I've never had accurate physics code in LT. I mean, the linear part was accurate. Kind of. But angular dynamics have always been extremely hacky in LT. But those days are over, and we now have a genuine physics integrator with accurate angular dynamics (i.e. applying impulses at various points on objects results in correct torque & force). Honestly it's not really a big point, as you likely won't notice many cases of fancy angular dynamics in LT. But I think the takeaway is that this stuff was written in Lua (yes, the physics step is in Lua...!) and runs well within the acceptable range for us to support a massive simulation.

I suppose it's also worth mentioning that mods could certainly leverage this to 'go further' with the physics in LT. You all know I'm a fan of realism right up until it starts cutting into fun, which, in my experience, happens quite early-on. I don't intend for ship thrusters to use real impulses, because I don't want players to have to spend hours trying to properly mass-balance their ships to get inertial characteristics that aren't impossible to fly with. But for those of you who love that good old abused phrase "real Newtonian physics" -- well, mod away. I already wrote the real physics for you ;)

Adam's BSPs are Stupidly-Fast

Several times over the past week I've had to turn around and tell Adam that his BSPs are just grossly over-performant. In our current LT sandbox, we can click objects to select them, view data about them, orbit them with the camera, etc. Selection requires raycasting the scene, which, in turn, requires raycasts against the BSP trees that provide the acceleration structures for meshes. Currently I am not using a broadphase structure, so we are literally raycasting against every single asteroid / ship in the scene. Really, it shouldn't even work. It should just melt the computer when you have thousands of complex objects in the scene. But it doesn't. In fact, the fps hit is so low on my machine that I forget how negligent I'm being.

The biggest place that raycasts are used (biggest in terms of 'most likely to cause performance problems') is in checking to see if a projectile weapon hits an object. Pulses, beams, railguns all use raycasts. Missiles are slightly more complex but still do some raycasting.

I guess where I'm going with this is: we can probably support like 10000000 million projectiles in the world at once. Or something like that. Basically, REALLY BIG BATTLES. 100 ships is going to be a cakewalk, apparently :shock:

Josh's Nebulae are Stupidly-Fast (and Pretty!)

I spent some time after-hours one night revisiting my most-prized algorithm: the nebula generator. I made it roughly 10x faster while also improving the visual fidelity AND making the generator more consistent, such that it yields less "really bad" systems. It was a fun night. We all know I'd die if I went more than half a year without touching 'nebula.glsl' :roll: But seriously, I'm happy about this, because nebula generation was one of the most expensive parts of system generation (by quite a lot). It's now much more reasonable, especially on less-powerful GPUs.

Render-Time Interpolation of Simulation State

We implemented it. This is a major technical component of games that need to have tight control over the game world simulation (like those that want to simulate universes). I've spoken about this in devlogs long, long ago. The idea is that simulation of the game world happens at a fixed rate, independent of the frame rate. This makes simulation more stable and typically more performant as well. It also brings a major challenge: your simulation state is no longer in-sync with your render loop, so now you have to 'guess' what the simulation state would look like at the current render time. To do so, you simply keep one frame's worth of historical state, and interpolate between the last and current state to obtain a smooth, linear approximation that gives the player the perception that everything is nice and continuous on-screen.

Major game component, done.

Dev UI

We've built (well, mostly Adam) a really handy developer UI to help with poking and prodding the game as we work on various parts of it. We decided that the real UI should be one of the last things we do (this was guided by the realization of how much time I've sunk into UI that later became useless when underlying game mechanics changed). Until then, though, we've got some nice programmer-artsy widgets to see what's going on.

Multiplayer: Just for Fun

Last Friday I implemented multiplayer. Just for kicks :ghost: Don't get excited (or worried), it's not a feature and we're not shipping with it. By multiplayer, I mean I implemented a very naive peer-to-peer network architecture so that Adam & Taylor (a good friend/fellow programmer at the tech park) & I could all throw asteroids around in the same star system. We would find the biggest asteroid that we could in the system, then set our cameras to orbit it, then fight to push/spin it in various directions with impulses. Not exactly MMO-of-the-year material, but it works.

So, the UDP socket API in our engine works, and I've proven that it's possible to implement (some degree of) multiplayer with it. I hope the LTMP modding team is ready! :V (Also, I was on linux and they were on windows, so it works across different platforms, as you would expect of a decent implementation.)

---

Conclusion:

Everything is coming together. The road has been thoroughly and meticulously paved, after oh-so-many setbacks, and we're FINALLY getting to spend almost all of our time in Lua at this point, just pushing forward...which is a really good sign! Game constructs are being built and carefully probed for performance along the way, ensuring that everything is done 'the right way' so that (ideally) we don't end up with any nasty performance-related surprises in the end. That which is being built is being built to last. Thus far, all of that effort we sunk into performance is REALLY paying off, because the work is moving super quickly.

I know it's probably difficult to appreciate how much progress the above points represent -- especially when screenshots don't look appreciably different than they did several months ago in the early days of LT in LuaJIT. But if you've been reading, you know the truth: under the hood, it's a whole different ballgame. This time, the muscle under the hood is enough to get us to release. I'm genuinely excited about it (and let me tell you, it's not easy to excite someone who's spent the last five years doing little more than staring at vim :lol: )

From here, we just keep at it. Every day, more game. Every day, more LT. Then one day...

...you know :)

PS ~ Some eye candy just because. Nothing you haven't seen before -- pretty nebulae, decent asteroids, bad ship algorithms...probably too much bloom :oops: But I know you guys require food every now and then :)

Limit Theory Devlog Gallery ~ August 31, 2017

Image

Image

Image

(More in the gallery)



Link to original: viewtopic.php?f=30&t=6281

((Talvieno's note: There is going to be a Kickstarter update very soon. I was initially going to wait for it and put them both in the same post, but I've opted to post this now, and then I'll be making another post with a copy of the Kickstarter post when it arrives.))
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: The Limit Theory Email/RSS News Thread

#34
Monday, October 2nd, 2017

Ahoy there! I know it's been a long time since Josh made a post, so I'm stepping in here for a moment with one from Adam, just to show progress is still being made. For those who don't know, Adam is one of the programmers Josh hired, and this is his first major post. We'll get back to you soon with more regular updates from Josh, but in the meantime, hopefully you can enjoy this one. :)
AdamByrd wrote:
Mon Oct 02, 2017 11:31 am
Hey guys. I'm going to try something new for a bit and see how it goes. In between Josh's bigger updates I'm going to start posting smaller updates. These mini-updates will focus on the specific problem I'm solving at the moment. The goal is to go deeper into detail than the bigger updates normally do and is often going to be centered around code architecture since I've been spending a lot of time there lately. They will also be more informal: I'm going to write them without much revision and without the flair that generally comes with Josh's writing. This is mostly just laziness that I'm selling as time savings and pragmatism. I also imagine this being a little more interactive than the usual updates, so feel free to ask specific questions or tell me what you'd like to hear more about and I'll do my best. Off topic stuff is fine as well as I realize you guys don't know a whole lot about me yet. What I won't be covering is what Josh is currently working on. For one I don't want to steal his thunder and two I want to ensure there's still new information in the regular updates. Onward!

Recap
First a bit of history. My primary goal since teaming up with Josh has been to get to gameplay as soon as possible. In a particularly productive team meeting early on this lead to the birth of Lil'T. The original goal was to have a single, playable system in a week. Something basic with player controls, a nebula, and some asteroids. This was back when I had just finished the BSP implementation. We had been working solely in the engine and didn't even have a separate (code) project for LT. I didn't know any LUA or how to start writing gameplay that leveraged the engine so this was a good motivator to start knocking down those barriers. I think it ended up being about 2 weeks before we really got to the point we wanted to be. I have to say, hooking up the BSPs for the first time and being able to click things in the scene felt really damn good.

Now, as much as I want to get to gameplay, the second major goal is making sure the code we write is shippable. It can't be prototype stuff that's going to need to be re-written. It doesn't have to 100% perfect and complete either, but it needs to be written in a way that can expanded without needing to be refactored. In getting the first phase of Lil'T running it became obvious that the infrastructure to support LUA needed to be in before going further. That took another few weeks and you've likely read about some of that in the past couple of updates. During that work we spent a lot of time profiling, testing, and tweaking things and it became pretty obvious that being able to do some of that while the game was running would save us a bunch of time so I set to task on the debug UI.

With the dev UI in a good spot we're shifting back to gameplay again. We're likely going to do this 1-2 combo of sprinting on gameplay for a bit then stepping back to improve the infrastructure supporting it a few more times. Often we can't really tell what the scaffolding needs to look like until we have concrete gameplay as a guide. We spent a couple of days talking through how we wanted to simulate entities. Since they have one-to-many parent-child relationships (a ship is the parent of it's turrets), the game objects form a tree structure. We compared hierarchical updates using depth first and breadth first traversal. We compared job-based (run on all entities) and event-based (build a list of entities that need to be updated). Ultimately we settled on event-based updates with the ability to create jobs within that framework.

Input vs Simulation
Once we nailed down how we wanted updates to be processed, input became the next target. We ended up with a lot of code that looks like this:

Code: Select all

function onUpdate (dt, hasFocus)
  if hasFocus then
    if PHX.Mouse.Pressed(PHX.MouseButton.Left) then
      self:fire()
    end
    if PHX.Mouse.Pressed(PHX.MouseButton.Right) then
      self.camera:setTarget(self:getPick())
    end
  end
end
(Note that I'm being loose with the term 'update' in the text (but not the code). The core game loop is essentially simulate, update, draw. Simulate is the fixed interval simulation step. Update happens right before draw and is for things like animation and interpolation. Draw is for, well, drawing. Also note that handling input in the update step is wrong and will be changed eventually. We aren't currently buffering input and since the simulation rate is much lower than the update rate we miss a lot of input when handling it in update at the moment.)

This isn't great. Having to manually track, pass, and handle focus in every single location that processes input is going to lead to a lot of mistakes and unnecessarily obtuse code flow. We knew this wasn't going to last long as it was being written and it needed to be addressed soon, before enough incorrect code built up that refactoring it would be a big task.

Josh and I talked through this quite a bit and his idea was to treat the game window as a fairly standard UI. The GameView would just be a UI widget that displays the final, rendered view of the simulation. The simulation becomes an autonomous thing that marches through time, updating its own state without any concept of a camera, a player, or input. UI widgets will provide visible controls and input handling that inform the simulation of requested state changes. The GameView decides how it should render the view of the simulation.

It took me a bit of time to understand the implications of this approach. It sounded a bit odd at first, but the more I worked through it, the more I was able to see the large number of subtle benefits.
  • The GameView gets the full benefit of the automatic layout functionality of the UI. Debug and gameplay UI elements can push the GameView out of the way in addition to just overlapping it. If you've noticed how the debug UI overlaps gameplay that's actually quite annoying. It's nicer if the debug window is able to make the gameplay view narrower and slide it to the right so they are side by side instead. In general it means we can allow the game view to be resized, moved, snapped, etc very easily and naturally. We can iterate on UI layout faster, allow players to customize their UI easily, and handle window resizes without any added work.
  • Multiple GameViews can exist (think radar, scanning, minimap, whole-system views, etc).
  • Input focus is already handled cleanly. The UI tracks what your mouse is over, which widget has focus, is being dragged, etc. It's trivial to extend this to have separate mouse and keyboard focus, hard and soft focus, separate onInput functions for letting only the focused widget handle input, and whatever else might be needed.
  • The amount of branching in gameplay code drops significantly. We don't have to check for focus or input because that gets hoisted to the highest level and doesn't need to be repeated all over the place. Even with our tiny system, the amount of code and complexity immediately dropped.
  • The simulation takes on more of a client-server model. Input is handled in the 'client' which then requests state changes on the 'server'. Validation in the simulation all happens in one place. We end up being forced to build a clean API in some sense.
  • Concepts like control schemes become much easier to think about and implement. Since each form of input is going to be a separate UI widget completely changing how the game controls just means creating a new widget and adding it to the UI hierarchy.
There's probably a lot more nuance that I'm leaving out. Anyway, we're currently using the dev UI to handle this. We didn't originally intend for the dev UI to be the final UI system but the performance is good enough that it's actually a possibility. It's powerful and easy to use. However, if the dev UI is swapped out in the future it's not going to affect the way the gameplay code is written so it's natural to move forward with it for now.

Of course, making this change meant a decent amount of refactoring. I had to pull all of our input handling out, decide where it went and reimplement it in the new system. I ended up with 3 new widgets: GameView, DebugWindow, and DebugPicker. I also reimplemented the data inspectors Josh added a while back so we can actually walk through the entire game state from UI windows. I also took the time to add a bit more UI polish: added stretch weights to grid columns/rows, fixed subtle positioning errors in labels, added alignment to labels.

This is mostly under-the-hood work, but I figure I can at least drop a gif of the GameView being animated smoothly: https://i.imgur.com/EkASTsN.mp4


This update is a bit higher level than I'm aiming for in general, but it's not a very involved or probably interesting topic and it's only a few days worth of work. Let me know what you guys want to hear more/less about so I know where to focus in the future posts.

Cheers
Link to original: viewtopic.php?f=30&t=6310
Post

Re: The Limit Theory Email/RSS News Thread

#35
Wednesday, October 4, 2017

Hey again everyone! 'Tis once again the season for Josh to apologize for waiting too long to post a new devlog... :ghost: Honestly, time just flies when you're having fun working on LT. And we've been doing a lot of that.

Talv, in his usual beyond-his-years Talvish wisdom, has advised me that it's more important to be consistent with devlogs than to be exciting, flashy, etc. I feel like I've heard that from you guys before. At some point, it may actually sink in to this thick head of mine :oops: At any rate, I'm going to turn over the 1-hour hourglass (that's not redundant because I also have a 30-minute 'hour'glass...) on my desk and force myself to finish this log before the last grain hits the bottom.

To accomodate the expedited writing process, I'm going to bullet-point the list of what I've been working on, then expand on a single element of focus (whichever I feel is the most important at the moment). We'll see how it goes!



Since Adam has already been kind enough to give us quite some detail on what he's been up to, I'm just going to go over my own work. So...over the past month, I've:

  • Implemented several new components for entities, most of which are in preparation for / in service to implementing gameplay logic
  • Implemented a mod-loading system that performs validation and loading of isolated mod files, which of course means that I also...
  • Defined a mod file format and started writing logic in mod form; ideally, most if not all of our actual gameplay logic will be written like this, both for our own convenience in being able to easily plug/unplug pieces of gameplay, as well as to ensure uniformity with respect to 'built-in' content vs. mod-added content
  • Implemented an event-handler architecture for effortlessly broadcasting events as well as hooking listeners to respond to said events
  • Worked with Adam to implement first pass of 'job' system, which is, essentially, the 'code' analogue of an ECS
  • Implemented several jobs / pulled much of the existing LT logic out into jobs, resulting in isolated, efficient, and easy-to-reason-about logic
  • Implemented (naive) collision detection and resolution (as a job) -- had lots of fun with large quantities of asteroids bashing into one another :D
  • Tackled many, many performance optimizations accross the board, attempting to ensure that we stay lean-and-mean as we push more content into the game; some examples are LOD meshes & automatic generation thereof, fast font glyph caching, fast vertex AO computation for SDFs, and more...
  • Explored building a simple editor program for expediting the process of adding components, entities, and logic + rolling those changes into a mod file and testing it in-game (heavily-inspired by the TES games and the legendary 'TES Construction Set' from back in my Morrowind modding days)
  • Began work on version 2 of the jobs system, which will, hopefully, unify how we write the heavy, running-almost-all-the-time gameplay logic and the sparse, happens-only-under-specific-conditions gameplay logic
  • Got the 64-bit Windows version of LT built & running



Jobs and Gameplay Logic

With our CTypes system firmly nailed-down and providing an excellent solution to how we handle data in the game, we're now looking to solve the equivalent problem with logic/functionality/code. After all, there is only data and function, nothing else :) With entities, we asked the question: how can we efficiently, easily, and extensibly define the format and manage the lifetime of the data that keeps track of an entity? How can we do so in a way that will support many diverse types of entities, while also supporting the potential to have hundreds-of-thousands of these entities in play, and perhaps even thousands or tens-of-thousands going into and out of existence in a matter of fractions of a second?

To those questions, we answered and answered with vigor in the form of an ECS, which became an ECS v2, which became CTypes.

Now we ask the analogous questions with respect to logic: how can we efficiently, easily, and extensibly define the format and manage the lifetime of the logic that runs on and creates the characteristic behavior of an entity? How can we do so in a way that will support diverse types of entity behaviors, and potentially hundreds-of-thousands of such behaviors in play at once, and potentially tens-of-thousands of these behaviors stopping, starting, or being replaced with a different behavior at any given moment?

I've touched on partial answers to this question in previous logs. Part of the answer is: exploit the sparsity of game logic to do 'as little work as possible' at every moment of time (note the symmetry with how we store 'as little data as possible' in CTypes by using native memory, pools, static typing, etc.) This means, for example, that an AI player who is travelling to a location does not enter a state where the script constantly checks whether the player has arrived at the destination. This is wasteful and fails to exploit the sparsity of the situation (the probability of arrival is exceptionally low on any given frame). Instead, the script informs the engine of the specific condition(s) upon which it should be resumed: "Dear engine, I am writing to inform you that I no longer require CPU cycles until my client, Sir NotEdisonTrent III, aka 'Trent', arrives within 100 metres of the asteroid @ memory address 0xA513701D. Alternatively, if my client should be the victim of an attack, I would like to be notified in spite of not having arrived at said asteroid. Thank you for your time, see you in a few billion cycles. Love, script/ai/TravelTo.lua." If the underlying logic engine is clever with respect to how it services such a request, then the difference here is the difference between being able to run a few hundred AI units vs. being able to run a few hundred thousand (not that we need to run a few hundred thousand -- but keep in mind that if we consider all the other logic going on in the game, then indeed, we are looking at on the order of a few hundred thousand bits of logic). It's an orders-of-magnitude subject, like many things in computing.

We already have a job system for easily running logic on large batches of entities that is very cache-friendly as well as easily-parallelizable. At the moment, I am working to fill the final piece of this puzzle: enabling said system to accommodate highly-sparse logic like the example above. The good news is that I have already designed (on paper) the architecture for the system. I'm working on the implementation as we speak (well, I mean, by the time you read this I'll be back to working on it)! Interestingly, implementing such a system is, in many ways, symmetric to designing a robust OS kernel, as many of the concepts have parallels in OS design: a piece of logic is a thread, sparse logic is a thread that waits on condition variables (a fundamental synchronization primitive used in low-level kernel / driver implementations), thread groups are woken by condition variable signals / broadcasts, a job is a process, each instance of a job is a thread, job state is thread-local storage (TLS), ...and so-on (of course it's much easier to do for a game than for an OS, don't be alarmed by my comparison!) Lucky for me, I did manage to pass the operating systems course at uni before dropping out ;) (Side-note: our professor was the guy who made one of the first scripting languages, Tcl. He was the best prof I ever had.)

An infrastructure for handling game logic is the last remaining element that I see between where we are now and a time where we can iterate wildly on shippable gameplay code. It's all very exciting :D



Success! Would you look at that? I finished just in the nick of time (ok, maybe by the time I proof-read and actually post, I will have gone a few grains over :roll:). I realize that you all probably would like more details on some of the other elements, but I'm clearly unable to give 'brief' summaries of things when I'm excited about them. And I'm excited about a lot of those bullet points. At least this way I'm able to write logs in a reasonable amount of time while still providing juicy details + a high-level picture of progress.

I really would have liked to explain a bit more about my work with the proof-of-concept editor, but alas, it would require another turn of the hourglass. For now, I just want to mention that it is being 'mostly shelved,' as the goal was to determine whether the cost-of-implementation of such a tool would be less than or greater than the time saved developing. From my exploration, it seems the cost-of-implementation would not justify the LT-dev time savings, so we will most likely not see continued work on the editor. I can imagine some pragmatists out there cheering and some dreamers out there crying :roll: :ghost: Well, come on, we have to save some things for LT 2, right? :lol: That being said, it still has some useful features: we can use it to easily get output from LuaJIT tracing & assembly dumping on Windows, where it's more difficult to do so than on Linux.

Now then. I'm going to hold myself to posting the next devlog next Friday. Period. Next Friday I will do the same thing that I did today: turn the hourglass, and write. Talvieno is right, as always: consistency is the key. And if I just wrote a devlog more often, I wouldn't have to squish so darn much into one anyway!

Oh dear. It appears next friday is the 13th. Ha. Oh well, numbers are just numbers, and, after all, 13 is both a prime AND a member of a twin prime pair, so I don't see how it could possibly be unlucky. See you all on the 13th... :ghost: :wave:



I took a few screenshots to accompany the wall of text: Limit Theory Devlog Gallery ~ October 4, 2017

Wireframe view of an asteroid field, where you can see the LOD mesh optimizations (notice the wireframe density remains approximately-constant in screen-space, despite asteroid size or distance. Constant screens-space detail is 'ideal' LOD.)

Image

I wrote a simple mod to shoot asteroids out of my ship. I used it to test the new collision detection & resolution job. It's fun blowing apart a big sphere of asteroids with a spray of little ones :nerd: (It's kind of hard to see without an animated GIF, but I'm not nearly as good as Adam at making those..)

Image

Using Plume (the concept LT editor) to get LuaJIT assembly dumps on Windows:

Image




Link to original: viewtopic.php?f=30&t=6315#p155467
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford

Online Now

Users browsing this forum: No registered users and 3 guests

cron