Monday, April 3, 2017
Posted: Mon Apr 03, 2017 6:33 pm
Monday, April 3, 2017
Happy April! Boy was it a long week. So much stuff happened it's hard to remember it all, but that's definitely better than the alternative. This week was all about two big pushes toward the completion of two big systems: the entity component system (ECS), and the physics engine. Neither was finished, but major progress was made on both fronts.
---
Entity Component System: Just to quickly recap, this is the part of the engine that powers all objects in the game (hence, the part of the engine most relevant to gameplay). It defines how the data that describes objects like ships, stations, colonies, asteroids, etc. is defined, managed, and altered over time in response to gameplay. From the point-of-view of most would-be LT modders (as well as Gameplay Josh), it's the most important part of the engine, perhaps followed by the UI system.
I spoke last week about the abundance of trade-offs inherent in designing a game engine's ECS. This week I continued to explore that abundance of hard choices by...implementing them. Describing all those choices in their technical detail would take too long and bore you all to tears. I started describing my current implementation in quite some detail in this log, stopped, and deleted it because I'm still too unsure about it With the lessons I learned from trying several different architectures, I started implementing a different one yesterday. But yesterday is...well, yesterday. Practical Josh, in general, does not feel confident about things being 'the answer' after only having had them around for a day.
But since I don't want to talk implementation details, I'll mentioned a new (to me) development paradigm that I've been using while playing with all of the options. Like many things I do, the idea sounds stupidly-obvious...but it almost felt like a revelation to me when I started doing it. I've been writing a good bit of gameplay pseudocode. Yep. Quelle breakthrough! But seriously:
It's so simple, incomplete, and really quite boring. Yet amazingly, it helps me in so many ways to think about how I should design the ECS from a practicality standpoint to do the stuff that I'm actually going to want to do. Even writing this silly code, I can get a feel for challenges that aren't obvious when designing from a blind, theoretical perspective. That code isn't real; it's not any real language, just my made-up pseudocode. But what we really want is to design things such that we can write gameplay code in Lua that rivals the simplicity of the code above.
Frankly, this has been equal parts difficult and rewarding. I'm bouncing back-and-forth between the highest-possible levels of gameplay coding (I mean, it's pseudocode, it's higher-level than the high-level gameplay code), and the lowest levels of engine architecture. It's mentally jarring to constantly bounce between those levels. But it's also rewarding to feel that these low-level choices are being made with very concrete goals in mind.
Physics: Great progress and several new structures/algorithms that I've never implemented before in LT, even back in the C++ engine. The LT Core picked up a lot of code. Octrees. AABB trees. K-d trees (unfinished). I have efficient code for building octrees & AABB trees from meshes or arbitrary volumes, as well as for raycasts. Full intersection tests will come soon.
On top of implementing a ton of physics-related data structures and algorithms, I did a lot of testing to determine which ones best solved the previous performance problems with the narrow-phase collision detection on large meshes. Not surprisingly, AABB trees are winning, at least with raycasts. I can do raycasts against a large, ultra-quality station mesh in something like 3 microseconds now. Although broadphase is not really a pressing issue and will come later, it was interesting to see that my tree construction code is fast enough to actually do the dumbest thing possible without killing FPS (that is, to reconstruct the whole acceleration structure every frame, inserting all objects in the system; this is incredibly stupid compared to real broadphases, which exploit temporal coherence to do dramatically less work).
Gratuitous screen from physics work:
And a few more if you want...
Lua & LuaJIT: As per my concerns last week, I did some investigation with respect to how to use LJ most efficiently. I used the profiling framework that I mentioned last month (and have expanded on greatly this week) to guide my investigation. I also started examining LJ bytecode dumps & trace information so that I can better understand how and where I'm losing cycles. The profiling framework has already proven itself to be invaluable!
I now have a better (but still expanding) understanding of writing efficient scripts, especially for gameplay logic. I implemented a system (in the LT Core) for script loading & running that allows better memory usage and performance than my previous monolithic architecture.
---
The coming week should be all about the entity component system. I've made great progress so far with trying a lot of implementations, and I'm getting ever-closer to a design that fits all of the requirements of LT. I'm putting this at max priority and giving myself full permission to do nothing but ECS work for the coming week. I want to start getting 'real' gameplay (not LTDemo gameplay) up; I'm ready for that pseudocode to become reality!
Hey everyone! Talvieno here, with another non-technical summary!
Josh has spent most of the past week or so working on the completion of two things - entities and physics. Just to recap, the entities are every object in the game - ships, stations, asteroids, planets, colonies - everything. The physics engine is how these objects interact physically with game space and each other - collisions, acceleration, motion and so on.
You may have noticed that he dumped a ton of "mining code" up there, and may have no idea what he's doing. Josh is basically doing his own version of "Rubber Duck Debugging". Rubber Duck Debugging is an old programmer method to figure out how to fix bugs: you talk to a little rubber duck sitting beside your computer and tell it what the problem is. In talking about the problem, you understand it better, and are often able to solve it much more easily than just from staring at the screen. Josh is writing pseudocode (fake code - "code" that's not actually code) to help him understand his problems, and finding it incredibly helpful.
The physics stuff is much more complicated, but in summary, Josh is working on collision detection. Basically, it splits up the game space into smaller, more manageable areas - splitting them up means the program has less space to check at any one time, which means a faster program. Josh includes a few visualizations of this in the form of shiny pictures.
As to the LuaJIT stuff - he's been experimenting with LuaJIT (the modding/scripting language) to try to find out how to get the most out of it - to squeeze out every last drop of efficiency for maximum framerate.
tl;dr: Josh has worked a lot in several areas the past week, mostly in solving complex problems. He's almost ready to start putting all the pieces of the puzzle together and make something you can actually play as LT.
Happy April! Boy was it a long week. So much stuff happened it's hard to remember it all, but that's definitely better than the alternative. This week was all about two big pushes toward the completion of two big systems: the entity component system (ECS), and the physics engine. Neither was finished, but major progress was made on both fronts.
---
Entity Component System: Just to quickly recap, this is the part of the engine that powers all objects in the game (hence, the part of the engine most relevant to gameplay). It defines how the data that describes objects like ships, stations, colonies, asteroids, etc. is defined, managed, and altered over time in response to gameplay. From the point-of-view of most would-be LT modders (as well as Gameplay Josh), it's the most important part of the engine, perhaps followed by the UI system.
I spoke last week about the abundance of trade-offs inherent in designing a game engine's ECS. This week I continued to explore that abundance of hard choices by...implementing them. Describing all those choices in their technical detail would take too long and bore you all to tears. I started describing my current implementation in quite some detail in this log, stopped, and deleted it because I'm still too unsure about it With the lessons I learned from trying several different architectures, I started implementing a different one yesterday. But yesterday is...well, yesterday. Practical Josh, in general, does not feel confident about things being 'the answer' after only having had them around for a day.
But since I don't want to talk implementation details, I'll mentioned a new (to me) development paradigm that I've been using while playing with all of the options. Like many things I do, the idea sounds stupidly-obvious...but it almost felt like a revelation to me when I started doing it. I've been writing a good bit of gameplay pseudocode. Yep. Quelle breakthrough! But seriously:
Code: Select all
/* Basic Mining AI Pseudocode. */
start:
self.clearActions()
targets = self.getSystem().getEntitiesWithComponent(Component.Harvestable)
miningTarget = argMax over target in targets of
target.getInventoryUsed() / self.getTravelTime(target)
self.pushAction(TravelToObject(miningTarget), align)
align:
/* Align myself 'tangent'-ish to the target. We could get way fancier here,
taking into account turret line-of-sight. Quick version is to just assume
being tangent will yield goood LOS for all. */
toSurface = miningTarget.getNearestSurfacePoint(self.getPos()) - self.getPos()
self.pushAction(AlignAxisYTo(toSurface), harvest)
harvest:
/* TODO: Maybe move around the target a bit so as to be less...boring? */
/* TODO: Fire each turret at a different (visible) surface point. */
for turret in self.getTurrets()
if turret.getType() == TurretType.TransferUnit
turret.fireAt(miningTarget)
self.notify(InventoryFull(self), full)
self.notify(InventoryEmpty(miningTarget), start)
self.notify(NotInRange(self, miningTarget), reposition)
full:
self.pushAction(DockAt(dropoff), unload)
reposition:
/* Calculate a point near the surface of the target such that all transfer
units are in range but I'm still a safe distance away */
self.pushAction(TravelToPoint(point), align)
unload:
self.transferInventory(dropoff, ItemType.RawMaterial)
self.pushAction(Undock(), start)
Frankly, this has been equal parts difficult and rewarding. I'm bouncing back-and-forth between the highest-possible levels of gameplay coding (I mean, it's pseudocode, it's higher-level than the high-level gameplay code), and the lowest levels of engine architecture. It's mentally jarring to constantly bounce between those levels. But it's also rewarding to feel that these low-level choices are being made with very concrete goals in mind.
Physics: Great progress and several new structures/algorithms that I've never implemented before in LT, even back in the C++ engine. The LT Core picked up a lot of code. Octrees. AABB trees. K-d trees (unfinished). I have efficient code for building octrees & AABB trees from meshes or arbitrary volumes, as well as for raycasts. Full intersection tests will come soon.
On top of implementing a ton of physics-related data structures and algorithms, I did a lot of testing to determine which ones best solved the previous performance problems with the narrow-phase collision detection on large meshes. Not surprisingly, AABB trees are winning, at least with raycasts. I can do raycasts against a large, ultra-quality station mesh in something like 3 microseconds now. Although broadphase is not really a pressing issue and will come later, it was interesting to see that my tree construction code is fast enough to actually do the dumbest thing possible without killing FPS (that is, to reconstruct the whole acceleration structure every frame, inserting all objects in the system; this is incredibly stupid compared to real broadphases, which exploit temporal coherence to do dramatically less work).
Gratuitous screen from physics work:
And a few more if you want...
Spoiler: SHOW
I now have a better (but still expanding) understanding of writing efficient scripts, especially for gameplay logic. I implemented a system (in the LT Core) for script loading & running that allows better memory usage and performance than my previous monolithic architecture.
---
The coming week should be all about the entity component system. I've made great progress so far with trying a lot of implementations, and I'm getting ever-closer to a design that fits all of the requirements of LT. I'm putting this at max priority and giving myself full permission to do nothing but ECS work for the coming week. I want to start getting 'real' gameplay (not LTDemo gameplay) up; I'm ready for that pseudocode to become reality!
Hey everyone! Talvieno here, with another non-technical summary!
Josh has spent most of the past week or so working on the completion of two things - entities and physics. Just to recap, the entities are every object in the game - ships, stations, asteroids, planets, colonies - everything. The physics engine is how these objects interact physically with game space and each other - collisions, acceleration, motion and so on.
You may have noticed that he dumped a ton of "mining code" up there, and may have no idea what he's doing. Josh is basically doing his own version of "Rubber Duck Debugging". Rubber Duck Debugging is an old programmer method to figure out how to fix bugs: you talk to a little rubber duck sitting beside your computer and tell it what the problem is. In talking about the problem, you understand it better, and are often able to solve it much more easily than just from staring at the screen. Josh is writing pseudocode (fake code - "code" that's not actually code) to help him understand his problems, and finding it incredibly helpful.
The physics stuff is much more complicated, but in summary, Josh is working on collision detection. Basically, it splits up the game space into smaller, more manageable areas - splitting them up means the program has less space to check at any one time, which means a faster program. Josh includes a few visualizations of this in the form of shiny pictures.
As to the LuaJIT stuff - he's been experimenting with LuaJIT (the modding/scripting language) to try to find out how to get the most out of it - to squeeze out every last drop of efficiency for maximum framerate.
tl;dr: Josh has worked a lot in several areas the past week, mostly in solving complex problems. He's almost ready to start putting all the pieces of the puzzle together and make something you can actually play as LT.