Sunday, August 4, 2013

Summary

(Warning: For some reason, I sense that my brain is pretty jumbled right now. I wouldn't be surprised if the thoughts come out jumbled as well )

Arrrggg I'm so ready for this simulation engine to be done!! But it keeps giving me grief. When will it end

The more I think about the abstraction engine, the more I realize that it's really a lot like energy conservation, and I'm trying to understand everything in terms of this. To build an abstract model of something, what we really want to do is find some calculable quantity, and then ensure that this quantity is "conserved" in the process of abstracting. In some cases it's stupidly-simple: thrust, mass, construction ability, power output, etc. But I'm having a very hard time understanding weapons. Again, this ties in to the attack event being the most difficult. I think weapons will pose the biggest problem for the engine - determining what to conserve and how to do so is hard when there are so many attributes that affect the behavior of a weapon. I'm starting to look at the problem of OOS simulation less as a problem of getting the logic correct and more as a problem of getting the abstract model of the world correct, which I think is an interesting take on it

At the same time, I'm still trying to understand the manner in which events "use" objects to bring about a change in the world. I've revealed that my intuition here was based on the concept of "power," and I think this is the correct direction. Today I realized that, when you get right down to it, the concept of being able to route power to different subsystems is really the exact same concept as being able to route your forces in a fight, route your construction modules to different tasks, etc. So it seems that this "power" concept is applicable at many different levels of the abstraction hierarchy. Unfortunately, the details of how the power allocators work is hurting me. You see, the "real" power distribution grid (i.e., the one used for giving energy to subsystems) has some time-varying logic attached to it. Ok, that's fine...but then, in order to be totally analogous to the way that "abstract power" is allocated, we need to have time-varying logic attached to abstract objects. This is suddenly seeming to be a big problem - not because it's hard to do, but because it means that the differential rollback system (as discussed in the last post) needs to keep track of ALL time-varying logic!!?? In essence, the entire game needs to be "undoable"

That's not something that I was anticipating, nor something that I would look forward to implementing. Is there a simpler way? Always the right question to ask...we'll see!

Every day this darn event is getting closer to completion, but it really seems like one heck of a long road, doesn't it??!

[ You can visit devtime.ltheory.com for more detailed information on today's work. ]

Monday, August 5, 2013

Summary

Nice!! First working implementation of weapon abstracting! I'm sure it's not perfectly balanced yet, but today I implemented a generic abstraction system for every "attribute" in the game, so now all "types" can abstract themselves (including weapon types), which, in turn, allows weapons to abstract themselves. Very cool! Like I said, it's probably not perfectly balanced, but the result does obey conservation of DPS, and the other properties are "similar" to whatever weapons were blurred together.

The result is that I can take a ship, a fleet, or a whole universe of ships and derive a single object to be used for simulation that approximates them - at least in terms of weaponry. The other attributes will be easier than weaponry, so I don't anticipate a problem

Sadly, there are going to be some real complications. Again, weapons and attacking will be the hardest part of the simulation to tackle...I'm pretty much sure of it. The biggest problem I face right now is missile launchers. I always said that I wanted to have launchers where you could swap out the loaded missile type. Since I love missiles so much, it's pretty much a requirement for me! I want to be able to carry swarms of tiny missiles to pepper smaller enemies with, but always pack a few ultra-high-quality sluggers in case I run into trouble. This kind of play style just appeals to me a lot. Unfortunately, I noticed (long before today) that this brings about a pretty serious problem: you can no longer say how much damage the weapon does, because it's now a function of the munition that's loaded. Before, this posed a challenge for the AI, because it would be difficult to know how much of a threat a given weapon is without knowing the munitions that are available to the carrying party. But now it's even worse, because the simulation engine really has no idea how much damage the the weapon can do - hence, it has no way to approximate these types of weapons at a coarse level. Furthermore, even if it took a heuristic guess at the average damage based on the munitions available to the weapon, we would still face the problem of how to correctly deplete ammo in a coarse simulation (if your missile-carrying fleet goes into an OOS battle, you would expect it to use up some munitions!) It's going to be a tough problem, but I'm sure there's a solution hidden somewhere

Range is the other big problem, and I knew it would be from the beginning. Again, it's one of these "discontinuity" problems, where the discontinuous game logic will cause problems for any kind of mathematical abstraction system that relies on some degree of continuity. The fact is, if all but one of your weapons has a range of 1km, but you have a nuke launcher with a range of 15km....this is an incredibly difficult situation to describe in a coarse manner. How powerful you are in battle is going to be completely dependent on the enemy's maneuverability and the range of engagement. I guess the general rule is that: the more specialized the situation becomes, the harder it's going to be to approximate. Interestingly, this has a direct connection to image compression! A very sharp, isolated detail is more likely to be lost under aggressive compression than a uniform color or simple pattern. It's really no different for game situations. Again, I'm sure there's a reasonable approximation hiding somewhere out there, but I'm pretty sure it's going to take some battle simulations to find it.

The really good news about simulation is that it can be verified easily, and I'm excited to try that when all of the logic is in place! The algorithm is simple: take a subset of the world, run the "full simulation" for some time. Take the same subset, feed it to the coarse simulator, and run for the same amount of time. Now translate the first world into a coarse version (so you have two coarse versions of the world) and compare! A measurement of how similar the two worlds are is a direct measurement of how accurate your coarse simulation is. Very simple This is going to be invaluable as I attempt to figure out the "best" approximations for situations like the one described above.

[ You can visit devtime.ltheory.com for more detailed information on today's work. ]

Tuesday, August 6, 2013

Summary

I took a good, hard look at items today - weapon types, ship types, thruster types, etc. At this point in the day I can't quite remember what spurred it, but I decided that things generally needed a more logical framework for computing the properties of types. Mass, for example, should be computed automatically from some volume approximation + density. Health should probably also be computed automatically. Value, energy requirement, ..etc. The list goes on - let's remove some degrees of freedom! I also refactored the notion of health and armor. In the new system, any physical object has some notion of the "material" out of which it is built. This will be used to determine both secondary attributes (like mass), as well as how the object reacts to damage. Armor no longer carries this type of property - it's just a bunch of mass made out of some material, which means it automatically mitigates damage in a certain way. I'm also interested in exploring whether the "material" of an object can automatically be computed from the "ingredients" of the blueprint or vice-versa. That'd be pretty cool - then there would actually be a logical correspondence between the raw material requirements of an item and the properties! You might come to recognize that equipment made from Derpium Ore is generally lighter than the rest (and that would carry over to weapons, armor, etc...anything made of Derpium ) The type system is looking cleaner and cleaner...I just need to keep removing degrees of freedom so that in the end we'll see more patterns and coherence!

There's still more "manual" coding than I would like when it comes to building an object based on the type of that object. For example, in the ship creation code, there's a line that tells the "health" component of the ship to set itself according to the max health given by the ship's type. This kind of manual code needs to disappear in favor of automatic interactions. What I'm starting to realize is that there's really a very direct correspondence between how my types (ship type, weapon type, etc..) are built and how my objects (ship, weapon, etc..) are built - and that this correspondence can be used to automatically configure an object. Today I implemented the code to do so, and now all I have to do to make sure that my ship has the properties of a certain ship type is - pass in the type! The components will then recursively examine the given type and configure themselves automatically. It's this kind of thing that is going to allow me to make the codebase maintainable in the end...so I'm always on the lookout

I finished "health" abstraction today - but it's not really much of an accomplishment, as everything else is easy compared to weapons! So now I think I have enough information to start writing some battle simulation code! But diving into it brought me to the next problem...

The real challenge I'm facing right now gets back to the whole (abstract) power distribution thing. Handling concurrent events and understanding how to divide resources among them is a challenge. In some sense, it's really the same question as: who do I target and which weapons do I use on each target? So you would think it wouldn't be too difficult. Just have some heuristic measurement of "importance" and allocate resources proportionally, or to the "most important" (depending on what kind of behavior you want). But..just the concept of concurrent reasoning in high-level logic is baffling enough. Thinking about distributing resources among some number of concurrent events...errr

Strong progress towards a simulated world, as always Hey, chin up, we've got abstract objects

[ You can visit devtime.ltheory.com for more detailed information on today's work. ]

Wednesday, August 7, 2013

Summary

Really didn't manage to get a whole lot done today I guess the combination of oversleeping, getting a new linux dev environment set up (I couldn't stand the lack of nice text antialiasing in the old one (Xfce)), and generally not having enough caffeine in my blood all conspired to prevent progress on the simulation.

Luckily, the later hours of the night brought about some considerable excitement that's worth noting! Previously, asteroid fields in LT have been pretty awful. Asteroids were treated just like every other object, which means, despite the engine being able to handle a few tens of thousands of objects happily, it just couldn't push enough data to make asteroid fields look vast. There's really no excuse for this: when you have a type of object that is very weakly-interacting, homogeneous, and needs millions upon millions of copies...come on, the answer is not to treat it like a strongly-interacting, heterogenous object of which there are a few hundred (ships, for example). Lazy Josh.

So I bucked up and wrote the system that should have been there from the beginning: an "infinite" asteroid field engine, where the asteroids are generated procedurally and streamed in / out dynamically based on location. The result is that, if you're in an asteroid field, you're always going to be seeing a LOT of asteroids in every direction. And the engine is actually doing less work, because it's not worrying about asteroids that are on the other side of the system! If you've seen the Infinity tech demo, you probably know what I'm talking about. When he flies through the rings of the planet, you can see loads of asteroids, and they're obviously being generated on the fly. LT now supports the same thing

Writing a feature like this reminds me of how insanely counter-intuitive games are when it comes to what takes the most time. I've been working on the "simulation" for like 2 to 3 months now, and you still see literally no sign of it in the game. Writing a hierarchical infinite asteroid engine took just short of an hour, and it's a night-and-day difference in the feeling of "vastness."

For those of you (and I hope it's close to all of you..) who have played Freelancer, the LT fields now feel similar. In FL there was definitely some kind of dynamic generating system, because the asteroids / debris seemed to go on forever! But I don't think it was hierarchical, because all of the roids / debris always looked roughly the same size. In LT we have the same system but hierarchical, so you will see a natural distribution (i.e. you'll see very large roids at a distance, just as you would expect). And you'll also see a lot further since this is 2013

Honestly, it makes such a huge difference...the game world feels a good order of magnitude more vast just with this tiny change. I haven't even started playing with density functions yet, but that's going to be another exciting aspect - carving out regions of higher / lower density using some good-old noise functions to give the fields variety

*sigh* ...always so hard to go back to the invisible work after a day like that...

[ You can visit devtime.ltheory.com for more detailed information on today's work. ]

Thursday, August 8, 2013

Summary

Continued working on the 'infinite' asteroid field engine, as per yesterday's post. Today brought a lot of efficiency improvements...I can now handle a pretty dense field at 60 fps!! Woo ("You're not actually going IN to an asteroid field!?")

As for the rest of the day, I decided (rather arbitrarily) to devote it to the upgrade-to-double-precision effort. Things are progressing nicely, and the overall low-level rendering architecture is going to be cleaner when I finish. This switch has forced me to think about some things that I've done stupidly in the past Things like representing orientations with 4x4 matrices, failing to make distinctions between directions and positions, etc. A lot of game textbooks advocate matrices when dealing with spatial operations. But when you really think about it, it's really not suitable as a data format until the very end. You've got two fundamentally different types of data that represent the spatial information about your entity: you've got a basis (the local X, Y, and Z axes), and then you've got an offset / position within the world. Two totally different concepts: one is concerned with directionality, the other is concerned with location.

I'm working hard to separate these concepts in the engine, so that the math becomes more intuitive and, more importantly, changing the underlying datatype used to represent position becomes more feasible (note that a double-precision engine does not need to use double precision for directions, only for position!)

This whole process has made me wish that I had made the distinction between point and direction earlier on. If I had forced a point / direction class distinction, I would have realized very early on that matrices are not suitable for general-purpose computation in the engine. I would have also had a mechanism for effortlessly understanding which parts of the code need to be translated to double precision, and which can remain single.

Ah well, live and learn! Hopefully we'll be seeing huuuuuge scales soon

[ You can visit devtime.ltheory.com for more detailed information on today's work.

Friday, August 9, 2013

Summary

'Twas a uniquely-painful day
As much code-ripping cleared the way
For a universe of twice the precision
Just make a slight incision
Here, here, and...oh goodness, oh dear
Look what I've done now
Hit an artery some how
With blood spilling forth in torrents
I think this level of pain warrants

A slightly shorter dev log than usual.

(But in the end, I did successfully uproot matrices from the rendering code, which is the first and perhaps most bloody step of the process (or at least second-most-bloody...))

[ You can visit devtime.ltheory.com for more detailed information on today's work.

Saturday, August 10, 2013

Summary

You know, I really just don't know how it happens. One minute you don't have a care in the world, just doing your little toy programming, mindlessly tapping the keys like a toddler. And then the next minute you're floating so high in the clouds of functional programming you can hardly remember what the ground feels like, because there are too many templates and typenames in the way. How does it happen

I think it went something like this:

"Josh's Slow Descent into Functional Hell"

• "Hmm, I wish the asteroid field rendered faster"
• "Hmm, look, the call to get the asteroid renderable is taking a while"
• "Hmm, I forgot to share the renderable generator across all asteroids, so it's probably hitting more cache misses than necessary with all those unique copies"
• "Hmm, how would I share it? This generic programming interface sure is a CreateReference(RefCounted(Cached(AutoPtr( pain in the rear ) ) ) ); "
• "Let's make it a bit cleaner."

At this point I've been working on improving LT's generic programming library for what feels like an eternity (in reality, I'm pretty sure it didn't exceed a day, since yesterday's dev log indicates that I wasn't working on it at that time...)

The results are awesome. Every few months my understanding of generic programming concepts gets stronger and I'm able to offer deep code discounts that make pricey functionality available to the masses with minimal code cost. I guess it was about time to upgrade again! This time I'm getting rid of so much annoying code!! It's a truly wonderful thing.

And what does it mean for you?? Probably nothing. But, as always, more powerful internal tools means more functionality in less time Like the component system, the generic programming system in LT has been put to extensive use all around the place, and I wouldn't be where I am today without it. Unlike the component system, I absolutely did not get the design right the first time. But I'm converging on something beautiful. Slowly but surely. A bit 1 / sqrt(x) ish, you know? Somewhat of a slow and painful journey, but in the end you know where you'll end up. And it's a beautiful place

PS ~ If you're wondering whether those asteroids ever did actually render faster, well, you'll have to hang tight just a little longer ~ my upgrades are finished but compilation is still not I'll know soon! Even if it didn't provide any improvements, the cleanliness was well worth the day's work

[ You can visit devtime.ltheory.com for more detailed information on today's work.

Week Summary

General Sentiment

It was really a nice balance of conceptual and practical this week. I'm pleased at the progress on the abstract simulation engine, yet also pleased with the concrete developments in the dynamic entity loading system (aka infinite asteroid fields)! Hope to keep up this balance throughout the month, so that I can have a lot to show on both sides of the spectrum at the end

Major Accomplishments

• Implemented DPS abstraction (for weapons)
• Implemented integrity/health abstraction
• Rewrote much of the item attributes system for more autonomy and less manual generation of attributes
• Implemented infinite asteroid field / dynamic entity loading system
• Continued work on double-precision upgrade
• Upgraded the generic programming library within LT (leading to more efficient coding in the future!)