Sunday, April 23, 2017
Posted: Sun Apr 23, 2017 8:19 pm
Sunday, April 23, 2017
It's not a perfect resolution of the grand war against entities, but this week has brought out the fiercest practicality that I can possibly muster, and for that I think we'll all be grateful. It is time to move on, dear friends, from this monstrous black hole of time and effort. Brighter prospects lie ahead.
---
Entity Component System: So...we're all tired of it by now, right? Me too. After many more hours drifted by this week with no sign of bringing me the end-all-be-all solution I've been wanting, I made what I consider to be a very practical compromise: I said "screw it." Since such crass language isn't allowed in my treasured notebooks, I instead wrote something marginally more profound: "design for the rule, not the exception." I'm sure someone has said that before. But now I'm saying it again. After enough neural burning on the problem, I decided it's rather absurd to continue this temporal bonfire (yes, LT development sometimes involves lighting time itself on fire) when the edge cases, not the majority ones, are holding back the design. I banged out an ECS that covers the 90% and gave myself permission to worry about thruster trails later. Here's my theatrical rendition of the past two weeks (Boss is, of course, practical Josh in disguise):
RenderQueues: A nice advancement necessitated by this week's ECS work was a core rendering feature missing from all previous version of the LT Engine: the 'RenderQueue.' Truthfully, the RQ type is little more than a 'modern' implementation of something OpenGL has had since the dawn of mankind (GL called them 'Display Lists'). Since modern man has decided that OpenGL's display lists were a bad choice to put in a 3D API, we've deprecated them (along with 90% of the GL API). Hence why I implemented them as a first-class engine feature (driver support for display lists, unlike with many deprecated features, actually is a problem). The feature is little more than exactly what it sounds like: a queue of rendering commands; essentially a data structure that encodes a miniature rendering program. This allows rich flexibility to specify rendering operations in Lua by building a RenderQueue, but allows the operations to be executed at the full, blistering speed of a tight C loop. In fact, I've some notion that, since RQs are basically implemented as mini bytecode interpreters (i.e., a decode loop with a switch statement), they might outperform my previous implementations of certain rendering operations (thanks to the fact that all of the relevant GL code is sitting right inside that hot loop; external dispatches that aren't GL calls are nowhere to be seen). But all that aside, the point is that I needed a flexible and fast way to specify rendering operations from Lua, and I've now got it.
I previously had an engine construct called "Model" that handled this type of thing; it was basically a list of Material/Mesh pairs, telling the engine what meshes to render (e.g., for a ship), and what material to use for each mesh (a material was itself a combination of a Shader and various shader parameters). Upon starting to write my Model class into the LT Core, I saw the light, realizing that all of these various constructs that I used to have (Model, Material, ShaderState (a bunch of shader parameters)) could be unified by a structure that encapsulates exactly what they are, down to the core: just some rendering operations and various shufflings of state. Now I can think of a Material as being a RenderQueue that binds a shader and sets up some shader parameters specific to that material. I can think of a Model as being an RQ that calls a sub-RQ corresponding to a material, then draws a mesh, etc. RQs can be nested within one-another to encapsulate the exact kind of sharing that I used to think of in terms of all these disparate constructs that I no longer need! Yay unification. Rendering life is quite simpler now. To think the GL guys had this insight in 3000 B.C. or whenever it was that GL came about
I have some dangerous notions about how we might get cute with dynamically building RQs to achieve various optimizations. I'll try to steer my mind away for now.
Mods, Content, The Works: As you'll recall, my objective at the moment is to write the Lua-side LT code in 'mod' format, so as to make life easy on myself while simultaneously ensuring that I know how everything is going to come together with respect to actual mods. I've made a good many changes and simplifications to the format this week to help myself pump out that core gameplay code. I was honestly getting a little overwhelmed and turned around with my previous system of hooks and so forth. I've simplified the mod format to be much more like a 'data description,' which makes things cleaner and easier to reason about (without any sacrifice in flexibility). This is one thing I really love about Lua. It's a beautiful language for describing data, given that the one and only 'advanced' datatype is a table, and data is, after all, just tables inside tables inside tables. Throw in the ability to bind table fields to actual code/functions, and you've got all the power you need to build a piece of 'data' that tells the engine how to handle whatever it is that the data is defining.
I now just have a 'content' sub-table within my mod description table that contains any number of entries. Those entries represent...well...the content of the mod! Right now, they can be entity type definitions (ship, station, planet, etc.), event listeners (to add functionality in specific places), component definitions (define new components for use with the ECS), and miscellaneous type definitions (I'm still not sure what to call things like 'Item' and 'Corporation' and so forth...they're datatypes of course, but don't need to be handled as 'entities' by the ECS).
Stupid example of what I'm working on right now:
Once all content is loaded from mod files (or base files, in this case), the engine pulls together all these bits of content and resolves everything to figure out exactly what every entity consists of, what events should be fired when, etc. Less manual labor than before. FYI the content creation code there is obviously placeholder
It has a ways to go to be sure; we can certainly pretty up the syntax later. Right now I'm interested in getting all of this stuff working. It honestly strains my brain a bit learning to think in this new manner of events and handlers rather than the old, linear update logic. But the straightforward 'dumbness' of the old update logic is exactly what killed performance, so it's time I learned to man up and think asynchronously.
---
In the coming week I have another LT showing at LSU on Friday, so my goal will be pretty straightforward: I'd like to see LTDemo running on the 'real' systems: using the C entity code, event-driven rather than monolithic update loops, etc. I'd honestly be happy to have that all working with the same level of functionality as before, because that would mean that I'd be in good shape to rapidly move forward with all systems sitting on a solid foundation. It'd be a major step. At present I'm not entirely sure how many man-hours of work that will entail, but I do intend to find out
Hey everyone! Talvieno here, with another week's edition of The Non-Technical LT Devlog.
This week, Josh focused hard on the whole ECS problem and got it solved - at least to the point that it's "good enough". He now has a single unified structure with which he can handle ships, stations, asteroids, planets, and other objects. This will make implementing content much faster. OldJosh is not pleased with this development and would rather he have a complete solution, but PracticalJosh - or NewJosh - has won out here. I think this is what everyone was hoping for.
RenderQueues: Exactly what it sounds like. Josh has implemented a unification of rendering commands that'll make it a lot easier to progress in the future. You can think of it like a to-do list for the engine's rendering component - the part that draws everything to the screen. Everything has its own little to-do list now, and can be modified/controlled via Lua - which means it can be modded.
Mods, Content, The Works: Essentially, this is Josh talking about how he's simplified the modding format to make it easier to use. There's not really much else to be said; I think that just about sums it up. At any rate, this next week he's going to try to focus on getting the LT code up and running for another showing at Louisiana State University.
tl;dr: Finished the entity component system, and set up the framework to make it much easier to add content, as he's getting to that point.
It's not a perfect resolution of the grand war against entities, but this week has brought out the fiercest practicality that I can possibly muster, and for that I think we'll all be grateful. It is time to move on, dear friends, from this monstrous black hole of time and effort. Brighter prospects lie ahead.
---
Entity Component System: So...we're all tired of it by now, right? Me too. After many more hours drifted by this week with no sign of bringing me the end-all-be-all solution I've been wanting, I made what I consider to be a very practical compromise: I said "screw it." Since such crass language isn't allowed in my treasured notebooks, I instead wrote something marginally more profound: "design for the rule, not the exception." I'm sure someone has said that before. But now I'm saying it again. After enough neural burning on the problem, I decided it's rather absurd to continue this temporal bonfire (yes, LT development sometimes involves lighting time itself on fire) when the edge cases, not the majority ones, are holding back the design. I banged out an ECS that covers the 90% and gave myself permission to worry about thruster trails later. Here's my theatrical rendition of the past two weeks (Boss is, of course, practical Josh in disguise):
And there you have it. Kinematics and a few other pieces of physics-related information, along with graphical information, live in C. The rest (requiring no per-frame updating) lives in Lua. Components are Lua-side. So really, it's just an ES that lives in C...an Entity System without the components, since I decided these are, in the 90% of cases, lightweight enough to be handled by Lua without issue. And so it will be until something starts causing major issues.Boss : "Build me that ECS."
Josh : "Aye sir."
< 1 week passes. >
Boss : "Where's my ECS."
Josh : "Working on it."
Boss : "Get it done kid."
< 1 week passes. >
Boss : "Where's my ECS."
Josh : "I have a lot of good designs."
Boss : "Neat. So where's my ECS."
Josh : "Well sir, currently none of them can elegantly accommodate the fact that thrusters have a key part of their rendering behavior tied to one little piece of state that should, in earnest, live on the Lua side, as it's really not 'intrinsic' to anything. Sadly, in the case of thrusters, there are enough of them that the data martialling will become problematic and result in framerate loss. On the other hand, it's difficult to see where this state should live on the C side of the system."
Boss : "...that right? So...just...thrusters..."
Josh : "Well sir the general problem is likely to crop up in a few other pla--"
< Boss raises hand so as to silence Josh. >
Boss : "It's all so...incredible. So fascinating. But see here, look Josh, let me ask you just this one thing."
Josh : "...yes sir?"
Boss : "WHERE. IS. MY. <expletives censored> ENTITY COMPONENT SYSTEM?"
Josh : *Quickly hands over imperfect but functional system*
< Boss storms away angrily. Cornelius Evazan enters from stage left. >
Cornelius : "You just watch yourself."
Josh : "I'll be careful."
Cornelius : "You'll be dead!"
RenderQueues: A nice advancement necessitated by this week's ECS work was a core rendering feature missing from all previous version of the LT Engine: the 'RenderQueue.' Truthfully, the RQ type is little more than a 'modern' implementation of something OpenGL has had since the dawn of mankind (GL called them 'Display Lists'). Since modern man has decided that OpenGL's display lists were a bad choice to put in a 3D API, we've deprecated them (along with 90% of the GL API). Hence why I implemented them as a first-class engine feature (driver support for display lists, unlike with many deprecated features, actually is a problem). The feature is little more than exactly what it sounds like: a queue of rendering commands; essentially a data structure that encodes a miniature rendering program. This allows rich flexibility to specify rendering operations in Lua by building a RenderQueue, but allows the operations to be executed at the full, blistering speed of a tight C loop. In fact, I've some notion that, since RQs are basically implemented as mini bytecode interpreters (i.e., a decode loop with a switch statement), they might outperform my previous implementations of certain rendering operations (thanks to the fact that all of the relevant GL code is sitting right inside that hot loop; external dispatches that aren't GL calls are nowhere to be seen). But all that aside, the point is that I needed a flexible and fast way to specify rendering operations from Lua, and I've now got it.
I previously had an engine construct called "Model" that handled this type of thing; it was basically a list of Material/Mesh pairs, telling the engine what meshes to render (e.g., for a ship), and what material to use for each mesh (a material was itself a combination of a Shader and various shader parameters). Upon starting to write my Model class into the LT Core, I saw the light, realizing that all of these various constructs that I used to have (Model, Material, ShaderState (a bunch of shader parameters)) could be unified by a structure that encapsulates exactly what they are, down to the core: just some rendering operations and various shufflings of state. Now I can think of a Material as being a RenderQueue that binds a shader and sets up some shader parameters specific to that material. I can think of a Model as being an RQ that calls a sub-RQ corresponding to a material, then draws a mesh, etc. RQs can be nested within one-another to encapsulate the exact kind of sharing that I used to think of in terms of all these disparate constructs that I no longer need! Yay unification. Rendering life is quite simpler now. To think the GL guys had this insight in 3000 B.C. or whenever it was that GL came about
I have some dangerous notions about how we might get cute with dynamically building RQs to achieve various optimizations. I'll try to steer my mind away for now.
Mods, Content, The Works: As you'll recall, my objective at the moment is to write the Lua-side LT code in 'mod' format, so as to make life easy on myself while simultaneously ensuring that I know how everything is going to come together with respect to actual mods. I've made a good many changes and simplifications to the format this week to help myself pump out that core gameplay code. I was honestly getting a little overwhelmed and turned around with my previous system of hooks and so forth. I've simplified the mod format to be much more like a 'data description,' which makes things cleaner and easier to reason about (without any sacrifice in flexibility). This is one thing I really love about Lua. It's a beautiful language for describing data, given that the one and only 'advanced' datatype is a table, and data is, after all, just tables inside tables inside tables. Throw in the ability to bind table fields to actual code/functions, and you've got all the power you need to build a piece of 'data' that tells the engine how to handle whatever it is that the data is defining.
I now just have a 'content' sub-table within my mod description table that contains any number of entries. Those entries represent...well...the content of the mod! Right now, they can be entity type definitions (ship, station, planet, etc.), event listeners (to add functionality in specific places), component definitions (define new components for use with the ECS), and miscellaneous type definitions (I'm still not sure what to call things like 'Item' and 'Corporation' and so forth...they're datatypes of course, but don't need to be handled as 'entities' by the ECS).
Stupid example of what I'm working on right now:
Code: Select all
return {
id = '_LTBE_SYSTEM',
name = 'Limit Theory Base Entities: System',
desc = 'Defines the System Entity',
author = 'Josh',
version = 0.01,
content = {
{
type = 'Entity',
id = 'System',
components = {
'CollisionDetector',
'EntityContainer',
'NaturalResources',
'Zones',
},
},
{
type = 'Listener',
id = 'System.onCreate',
fn = function (Game, e, seed)
print('Looky, a system!')
local rng = RNG.Create(seed)
e.name = 'Omicron ' .. tostring(rng:getInt(10, 20))
e.starDir = rng:getDirection3()
e.starColor = rng:getVec3(0.5, 1.0)
-- Populate the system with...you know...lots of stuff
end,
},
{
type = 'Listener',
id = 'Engine.onInit',
fn = function (Game)
local seed = 1337
local system = Game:createEntity('System', seed)
-- TODO: Implement Limit Theory
end,
},
}
}
It has a ways to go to be sure; we can certainly pretty up the syntax later. Right now I'm interested in getting all of this stuff working. It honestly strains my brain a bit learning to think in this new manner of events and handlers rather than the old, linear update logic. But the straightforward 'dumbness' of the old update logic is exactly what killed performance, so it's time I learned to man up and think asynchronously.
---
In the coming week I have another LT showing at LSU on Friday, so my goal will be pretty straightforward: I'd like to see LTDemo running on the 'real' systems: using the C entity code, event-driven rather than monolithic update loops, etc. I'd honestly be happy to have that all working with the same level of functionality as before, because that would mean that I'd be in good shape to rapidly move forward with all systems sitting on a solid foundation. It'd be a major step. At present I'm not entirely sure how many man-hours of work that will entail, but I do intend to find out
Hey everyone! Talvieno here, with another week's edition of The Non-Technical LT Devlog.
This week, Josh focused hard on the whole ECS problem and got it solved - at least to the point that it's "good enough". He now has a single unified structure with which he can handle ships, stations, asteroids, planets, and other objects. This will make implementing content much faster. OldJosh is not pleased with this development and would rather he have a complete solution, but PracticalJosh - or NewJosh - has won out here. I think this is what everyone was hoping for.
RenderQueues: Exactly what it sounds like. Josh has implemented a unification of rendering commands that'll make it a lot easier to progress in the future. You can think of it like a to-do list for the engine's rendering component - the part that draws everything to the screen. Everything has its own little to-do list now, and can be modified/controlled via Lua - which means it can be modded.
Mods, Content, The Works: Essentially, this is Josh talking about how he's simplified the modding format to make it easier to use. There's not really much else to be said; I think that just about sums it up. At any rate, this next week he's going to try to focus on getting the LT code up and running for another showing at Louisiana State University.
tl;dr: Finished the entity component system, and set up the framework to make it much easier to add content, as he's getting to that point.