Return to “[Archived] Daily Dev Logs, 2012 - 2015”


Week of June 15, 2014

Sunday, June 15, 2014


Playing With Planets

I mean, how could I not play with planets after finishing the planet type code yesterday? Surely this is no surprise :monkey:

At any rate, today I explored new surface patterns, color gradients, scattering tweaks, etc. in search of new and exotic planet appearances. I'll admit, it feels more like fun than work...but that's ok :roll: Really stumbled upon some beautiful-looking worlds today.

SMAA Custom Tweaking for Better Specular AA

It's been bothering me a bit lately how much specular aliasing we're seeing on things like small, distant asteroids. SMAA works almost magically well for traditional edges, but seems to have a harder time dealing with things like the single-pixel jaggies caused by physically-based shading.

Today I dug into the SMAA code to understand it a bit better, and ended up doing a minor rewrite of the last pass (SMAA is a three-pass ordeal) to perform a more aggressive blend using the detected edges. With this new blend I am able to significantly suppress the specular aliasing :) I will need to keep an eye on this to make sure it doesn't degrade quality in other areas (as I'm a bit surprised this wasn't used in the vanilla SMAA code), but so far things are looking quite a bit better! :geek:

Implicit Casts in UDF Function Resolution

Not a huge achievement or anything, but I mentioned the other day that I developed the technology for supporting automatic implicit casts in UDF. This is pretty much a necessity when you've got stuff like signed and unsigned ints, floats, etc. all floating around (teehee) in the same script. MyFunction(1) should pretty much just work, regardless of whether MyFunction takes a signed int, unsigned int, float, bool, etc. Perhaps surprisingly, that kind of thing doesn't just happen automatically, though. A compiler must recognize that the inner type (which is really an integer) can be automatically cast to those other types. In the case of a non-constant inner expression, it must even insert runtime code to perform that cast during execution (e.g. MyFunctionTakingAFloat(MyFunctionReturningAnInt(..)) needs extra runtime code that takes the return value of the inner function and casts it to a float before passing to the outer function).

That bit is done now! Just one more step forward :thumbup:

I didn't yet implement full overload resolution preferences (e.g., if two function overloads match and one requires casts while the other doesn't, the no-casting overload should be preferred without any ambiguity). That'll just take a few moments when I feel like doing it :geek:


June is picking up more and more speed, and I don't see any signs of that letting up. We've got a lot left to do, but we've also got some mighty powerful tools and tech to bring to bear on the LT universe :D
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford

Re: Week of June 15, 2014

Monday, June 16, 2014

Massive day.

Parameters in UDF Scripts

Implemented parameters today for external scripts, so that you can declare inputs to your script. Naturally, the engine will expect a certain parameter layout for certain types of scripts (for example, a lot of the algorithms will take an integer 'seed').

Pretty straightforward :)

Generating Algorithms as Script Calls

As expected, I am going to move forward with externalizing the 'content' parts of LT, and, in doing so, am going to externalize the content generating algorithms.

Currently, there's a fair amount of complexity surrounding the way that assets are encoded in the engine - in particular, a ship, for example, does not just have a model. It has a small piece of data that basically explains to the engine how to create the model for the ship. I spoke about this paradigm quite some time ago. It allows the engine to load and unload the actual procedural resources as necessary, all the while retaining 'instructions' for how to create anything that is needed.

Today I started to replace this paradigm with what should be the final form: calls to external scripts. Instead of instructions being directed at in-engine generating algorithms, the instructions that objects now carry around with them will be in the form of information about what external script to call and how to call it. Then, when the asset needs to be loaded, the engine simply executes an external UDF script to obtain the result. Makes sense, and is even more unified and elegant than what I had previously.

But that idea leads to many others... :geek: ...

Automatic Caching of Script Call Results

Here's the big one. This idea is a huge one. Today, in my quest for more polish, I kept thinking "man, it's annoying how when I first load the universe, I get low framerates while models are being generated and resources are being loaded." Part of the problem is that, while the really expensive assets are already being cached to disk (like background nebulae and planet surfaces), a lot of the generating algorithms are 'fairly fast,' and so I don't cache their results between runs of the game. Things like ship model & texture generation, asteroid models, etc. are not cached, so the GPU scheduler is running in the background when I first load the game. Although it's not terribly distracting, it causes a big hit on the 'smoothness' of the game, especially in the first 20 seconds or so, which, coincidentally, is often about how long I run the game when I need to quickly test something.

While thinking about the fact that I needed to add caching to the other generating algorithms like I had done for the expensive ones, I started to think a bit deeper. "Well, hold on...we really want these algorithms to be controlled through UDF so that they can be moddable...maybe I should be looking at caching the results of calling UDF functions." And that's when it hit me. By design, the results of any script function are constant given the same inputs and variable states (controlled randomness is always achieved in LT through seed values, so that the result is the same when given the same seed). At the top level of the script, the only inputs are script parameters, there are no external global variables. So this means that, necessarily, any script returns the same result if the script parameters are the same. Bingo. It means that we can automatically cache script results in a really generic way - we simply store the script name, a hash of the script contents (so that we can regenerate the results if the script itself is modified), a hash of the inputs (note: don't call me out on that one because yes, it's not actually that simple, but...for the sake of the dev log ;) ), and the output value. Then, when the script is called from the engine, we can automatically check to see if the result has been computed already for the given parameters.

The result? It would mean automatic, transparent caching of all generating algorithms. Sure, nebulae and planets...but also all models, textures, etc. All of the procedural algorithms would automatically cache themselves and execute lightning fast on the second go. Since the caching is generic, rather than specific, we can also implement fancy mechanisms later on - like evicting seldom-used elements if the script cache grows too large. This way, we'll automatically be deleting the detailed, procedural data from systems that haven't been visited in a long time. And it would all happen automatically and for free.

Now that's a mechanism I could get behind :geek: In fact...I already did! This has been implemented in basic form today, and I will continue to improve it as I replace the current generating algorithms with script calls.

Not only does this put the power of the script engine to great use, but it also encourages me to make sure that as many generating algorithms as possible are factored out to scripts rather than hard-coded, since I then get automatic caching. I need to externalize those algorithms so that I can tweak LT quickly (and so that others can mod it), but now I have another really good incentive to do so.

Huge stuff :D

Warp Tunnel Graphics

I'm just gonna go ahead and drop that subtitle here so that you know I wasn't working on scripting tech all day. And that's all I'm going to say about it ;)

“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford

Re: Week of June 15, 2014

Tuesday, June 17, 2014

[enthusiastic comment about having two great days in a row] :lol:

Content: AI Expansion Metaproject

The next step for AI content is expansion. We've got a (basic) 'monopolization' metaproject already working, wherein the AI attempts to use money to damage the competition. Next up, I will want to see the AI actively buying and outfitting new equipment to expand the scale of a project.

The great thing about this metaproject, though, is that it ties in to so many areas of gameplay. In particular, in order for expansion to be viable, it means the AI must know how to use production mechanics, acquire equipment from the marketplace, outfit their ships, etc. The day that we see a full 'mine ore -> refine -> produce equipment -> use equipment to expand mining operations' cycle will be a glorious day. We've got some work left to do (in particular, we need to wire production tasks into the universe's job cache, so that AIs will start to consider production as a viable project). But it's going to be fun work. I have started on it today :geek:

Polish: Decoupled Camera: Attempt #2

A brief history of Limit Theory's camera
In the beginning there was a classic, (coupled) mass-spring camera. No surprises there.
In LTP 1.1 we saw the first 'decoupled' camera in LT, which allowed capital ships to effectively aim turrets and take out fighters, even though their turning speed was too slow to bring the ship about.
A few months ago, I ripped it out because I was having reservations about the immersion quality of this type of camera. I resolved to keep a coupled camera and deal with the capital ship problem another day.
A month or so after that, I discovered a trick that made the camera motion feel a lot more like the Freelancer camera ("the golden standard").
But time passed and grew disillusioned and uncomfortable with a coupled camera.

Today, I have once again stood up and restarted the search for the perfect camera. The perfect camera, in my mind, would feel a lot like Freelancer's, in that it would be fast and responsive, yet smooth rather than rigid. It would gently accentuate motion rather than ignore or bind completely to it. The perfect camera would make capital ship aiming easy and intuitive. The perfect camera would make sense to a new player as soon as their hand touched the mouse.

I did not find the perfect camera today. But I'd venture to say that my second attempt at the decoupled camera is the best yet. It feels a lot like Freelancer when piloting small, nimble ships. It feels a lot like LTP when piloting bigger ships (which is good). It makes me feel more free yet immersed in the universe than ever. Again, it's not perfect, but it's a lot better :geek:

Tech: Stack-Based Variables in UDF

I spoke a few days ago about how variables should eventually be implemented to operate in constant time by using compile-time allocation of a fixed variable stack, rather than a dynamic map. Today I implemented this mechanism, so that script execution is, in theory, a lot faster when it comes to using variables.

Not particularly hard, but a fun exercise in compilers nonetheless :)


You'll notice today that I prefixed my entries with content, polish, and tech. I would like to challenge myself to do so for the rest of the month (that is, to allocate my time carefully enough to have some progress in each of content, polish, and tech every day). I'm sure I won't be able to do all three every day, but it will be a loose goal for me for the rest of June :) I can't afford to get sucked too far down any one rabbit hole!
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford

Re: Week of June 15, 2014

Wednesday, June 18, 2014

Third big day in a row. Brace yourself :cool:

Tech: Hierarchical Scenegraph

Well. I said it would happen someday. Today, for whatever reason, I just got tired of not having this nugget of beauty. I got tired of not being able to think of zones hierarchically. I got tired of not being able to recursively dive into the system map without wasting time processing every single object in the system. I got tired of having to make sure that all child objects are always registered with the container in which they live, instead of elegantly being able to add and remove them as I please, having the hierarchy do the rest of the work.

When I had an idea concerning zones, dynamic content generation, and culling, it was just the straw that broke the camel's back. I had to have my hierarchy. And so I did :geek:

I won't say much more about it, because the important thing is: lots of blood, many battles, but ultimately, a victory :clap: :ghost:

Content: Zones and Dynamic Content Generation

A while back, I showed some technology that allows the engine to create infinite asteroid fields in all directions. Today I still use it for smaller debris scattered throughout the system (because I've grown tired of being in one massive asteroid field).

Today, I realized that this concept should be tied to zones. Instead of thinking of the system as generating an infinite expanse of dynamically-generated objects (asteroids, ice, etc.), we should think of zones as doing that! Instead of taking the system and masking space with a density function to carve out zones, we should just have explicit zones that handle all the work of generating whatever content should be inside of them.

Asteroid field zones automatically populate themselves with asteroids as you approach them. The same applies to ice fields. Debris fields. Planetary rings. Clouds of magical space gnomes. Anything you want, really :D Think of a 'zone' as being both a spatial boundary as well as a generator of dynamic content to be explored. Not only do we save a lot of processing power by considering all zones to be dynamic generators (which can be loaded and unloaded freely), but we also gain freedom to paint a more interesting universe with higher object density and a lot more stuff to be explored!

This idea really requires the hierarchical scenegraph to operate effectively, which is why I was so keen to finally tackle that problem today.

I haven't finished with the new zone stuff yet, but I'm super eager to hop to it. Coming at you shortly :geek: I think this is going to allow me a huge amount of new freedom to create interesting, fun-to-explore system layouts without having to worry so much about performance!

Polish: Fixed Particle System, Faster Rendering

With the new scenegraph structure comes a few bonuses, almost for free.

I implemented a visibility list during the rendering phase, so that the scenegraph only needs to be traversed once during rendering to build the visible object list, and then all other rendering operations use the list instead of the scenegraph. This is significantly more efficient than the previous technique, especially when large numbers of objects are not visible :thumbup:

I've also finally managed to fix the particle system, which was having an issue due to the nature of the previous scenegraph structure and the way that frame interpolation works. They are now smooth and working again :)


The real question is: for how many days can this streak continue?? What will be the end result? Limit Theory 2.0 perhaps?? :lol: :roll: A man can dream.
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford

Re: Week of June 15, 2014

Thursday, June 19, 2014

Ok! Finally a bit of a break from the crazy flurry of this week :) I'm a bit tired out by these massive dev'll try to keep it shorter this time :shock:

Polish: Real Thruster Trails

Back in the early days of the engine I used 'real' thruster trails - trails that actually followed the path of the thruster through space, and then faded. I took them out for...unknown reasons? Since then I've been using a straight, linear trail that doesn't actually follow the curvature of the flight path.

Today I randomly decided to try putting the real trails back in. I was surprised at how big of a difference this minor visual effect makes. Especially during dogfights, it makes all the difference in the world. Being able to see a bit of the enemy's flight path history (by looking at the trail) actually makes it easier to aim. It also just looks a lot cooler.

I've got some more work to do before they're polished enough to keep permanently, but I'm really liking where this is headed :geek:

Polish: More Decoupled Camera Tweaking

I have some new tricks today that I added to the 3rd-person camera to make it feel even better! I'm honestly feeling better than ever about the flight model and general feeling of flight / combat in 3rd person. Still have some room for improvement, but it's really starting to come together :)

Tech: Working on Dynamic Content Zones

As per yesterday's log, I'm eager to get true, dynamically-generated asteroid / ice / debris fields working with the power of the new scenegraph. I haven't been able to do it yet because I'm having trouble deciding exactly how I want to structure everything (in particular, I want to make sure that the content generation can be easily scripted so people can create new, interesting types of zones). Just need a little more time on this one :)


Very pleased with how June is turning out so far. Less AI that I had wanted so far, but there's still time left ;)
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford

Re: Week of June 15, 2014

Friday, June 20, 2014

Got it :cool:

Content / Tech-ish? : Dynamic Content Zones

We've finally got it. Today I ripped out what used to be the 'infinite asteroid field' technology and merged it into zone theory. We've now got zones that understand how to generate content on the fly, be it asteroids, ice, or gnomes. The zones also modulate the content with their shaping function, so we can have significantly more interesting boundaries...which brings me to my next point...

Tech: Arbitrarily-Shaped Zones

Today zones take on a new shape. Any shape, actually. Instead of ellipsoids or boxes or what-have-you, I implement generically-defined zone shapes using the same scalar field technology that's used to generate models, control AI behavior, etc. Makes total sense, right? Zones are shaped with a scalar field. This means it's easy to create a zone that looks like an ellipsoidal region. But it's also easy to create something that's shaped like planetary rings, weird, spindly asteroid field regions, etc. Whatever you want...if it can be written as a function (and trust me, it can), you can have it :)

The engine also now correctly uses these fields to determine when you are entering a zone, to send zone-related properties to the engine (like dust density), and to start dynamic content generation for that zone.


Absolutely thrilled with this recent work. Think I need several more hours just to play with it :) A big remaining question, however, is the LOD representation for dynamic zones, and that's no trivial problem. Can we find a general solution that would allow us to see planetary rings, ice fields, asteroid fields, etc. of arbitrary shape from a distance while looking decent? How could we do that? Since the zones are shaped by scalar fields, we could consider polygonizing the field boundary to create a LOD mesh for the zone (that's how models are already made). That's easy enough. It would give us a perfect mesh to represent the zone boundary. But then how would we texture it? An ice zone would need some kind of texture that fakes the look of big ice chunks at a distance. A planetary ring would...well, look a lot different. Is there a unified solution?

That's not really a question, though. There always is. You just have to find it :geek:

Oh June. :D
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford

Re: Week of June 15, 2014

Saturday, June 21, 2014

Hmm...for some reason, I just couldn't seem to get in the zone today (teehee) :|

Polish: Stable Dynamic Content

I finally managed to hunt down and kill a bug that's been plaguing the dynamically-generated asteroids for a long time. The bug was causing the contents of a cell of asteroids to change suddenly when the cell needed to be recomputed. The result was some pretty nasty asteroid popping when a cell changed configuration. Today it's been fixed, so no more popping of cell contents :) This applies to all dynamic zone content! Very happy to finally have that one squashed.

Tech: Profiling & Optimization

I spent some time today instrumenting more of the codebase for profiling, and, accordingly, hunting down some nice little performance gains :D Frankly, the game is running more smoothly than ever thanks to the dynamic zones (so that not all of the system content needs to be loaded and rendered at all times). But it's still always nice to grab a few free microseconds here and there :geek:


Alright, still a week left in June and there's a whole lot more that I want to see happen before the end. You know what that time ;)
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford

Re: Week of June 15, 2014

Summary of the Week of June 15, 2014
  • Implemented script parameters, automatic (implicit) casts, and stack-based variables in UDF
  • Finally implemented a hierarchical scenegraph! The LT scene data is now much more...structured :)
  • Implemented arbitrarily-shaped zones for easy creation of interesting zone boundaries (like planetary rings!)
  • Fixed and re-integrated dynamic thruster trails
  • Lots of polish / improvements to the 3rd-person camera
  • Automatic script caching theory
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford

Online Now

Users browsing this forum: No registered users and 1 guest