Return to “[Archived] Daily Dev Logs, 2012 - 2015”

Post

Week of October 13, 2013

#1
Sunday, October 13, 2013

Summary

Today I started down a strange and abstract path that I don't fully understand yet, but keep feeling a pull towards. I wrote some facilities that allow me (in one line of code ;) ) to register a function with the reflection engine. The registration provides the reflection engine with all of the relevant function metadata. This means that, for any given type (take, for example, an icon), I can get a list of every (registered) function in the engine that produces that type as an output. So the LT reflection engine is now equipped to provide not just type metadata, but function metadata as well! :ugeek:

I'm thinking that this is the final power boost that UDF needs to help me write everything that I need to write. If I can figure out how to marry this function registration mechanism with UDF, then UDF becomes much more than just a plain-old-data specification format. It becomes data and function calls, which means it can contain generating mechanisms rather than just straight-up data. I need that, because there are certain times when I don't want to just spell out all of the data. There are times when I need to call a function to automatically generate some of it.

This work is directly relevant to technology and item creation, which, of course, is the primary objective at the moment! The thing is, there's this weird middle ground that exists between technology and an item type - a middle ground which I am trying very hard to understand :think: There's like, a meta-item type, if you will. A type that specifies everything that one needs to generate an item type. For example, imagine a ship type. A ship type includes the model that is used to render that given type of ship. But technology does not directly specify the model. UDF should not directly specify the model. Both tech and UDF should specify the generating parameters that are then handed to a PCG ship generator function call. "Curviness," "repeated segments," "mirror axis," etc. Those kinds of things. Those are the things that you want to generate from a technology or read from a UDF file, but those aren't the kinds of things that exist in the actual ship type data. So there's clearly a translation layer in-between.

It's a bit frustrating, because I know I'm right on the tip of some major revelation. Not just concerning LT, but some major revelation in understanding code, data, life, and the universe. Something to do with "constructor" ...a word that's been on my mind a lot lately. Such a simple but amazing idea. Taking some set of data and constructing another piece of data from it...using code. In the above example, you can think of the ship generator as being a "constructor" for the ship type. It takes some set of generating parameters and "constructs" a ship type.

But you know, when you think about it, a ship type in and of itself is just the generating data for a ship...in the exact same way that the meta-type produced by technology would be the generating data for a ship type :shock: Goodness, it's really all the same thing, isn't it. I swear I'll go mad before this project is over. I can see it now...locked up in a padded white cell, shivering, sweating, yelling "data...function...data....function...data data...function data data...FUNCTION!!! ...no...DATA! YES!!! DATA. no..IT'S ALL THE SAME!! EVERYTHING IS A FUNCTION OR DATA!! but wait.....is function a data...I mean...is data a function...yes..maybe..NO..." :crazy:

:shifty:

[ You can visit devtime.ltheory.com for more detailed information on today's work. ]
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: Week of October 13, 2013

#2
Monday, October 14, 2013

Summary

Ok, so far doing a better job of balancing the visible and invisible progress this week! :)

First of all, I completed construction of my superweapon. Yesterday I had function reflection, today I have function reflection + parameter name information + nice automatic types that encapsulate all of the parameters of a given function. With this dangerously-powerful ability in hand, I could get entirely too cute if I wanted to, and do something like: typedef CreateShip::Parameters ShipType. Boom. Just totally eliminate all of the code surrounding my ship type class and replace it with automatically-generated code for what it really is: the generating parameters of a ship. But! I'm not sure I'm ready to get that cute :roll: The important thing is that I can use CreateShipType::Parameters to handle the "meta-item" to which I referred yesterday. So that question is solved and I now can approach the meta-types without actually having to go in and write more code :thumbup:

On the things-that-people-actually-care-about front, I got back to work on UI windowing today. Fixing side-docking and polishing a few subtleties. It's surprising how many subtleties go into making a windowing system feel "right" / intuitive. But I'm getting there! I also implemented window-on-window docking, so that windows will automatically snap to nearby neighbors. In fact, they'll do more than just snap - they'll actually attach themselves! It's pretty cool, because you can actually build your own UI structure out of windows by attaching them to one another (then when you drag one, you'll drag all of them :D). Neat!

Ah, and I also made a tiny but critical improvement to the cursor, which should totally eliminate cursor latency! Up until today I've gravely misunderstood the concept of software cursors. Up until today the cursor was always subject to a one-frame latency. Even at 60hz, the latency is noticeable. It's the difference between an application feeling slightly sluggish and feeling super-crisp. LT Engine applications now feel super-crisp, even without a hardware cursor, thanks to this revelation that eliminated the cursor latency :)

Seriously folks, I need to finish research. I keep pumping out new technology that is 'supposedly' going to help me with research mechanics but...it's time to get down to business. Show me the money. Josh, we need research. Stop talking about all this technical nonsense and just give us the research :monkey:

[ You can visit devtime.ltheory.com for more detailed information on today's work. ]
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: Week of October 13, 2013

#3
Tuesday, October 15, 2013

Summary

Totally side-tracked by graphics today! And that's never a bad thing :D

Toward the beginning of the day, I decided to toy with the bloom effect, just to see if I could improve it a bit. I've wondered for a while about some of the lighting effects, including bloom, in Skyrim - they're really lovely. Something I noticed after studying them for a while is that the bloom isn't constrained to bright colors. At the same time, it's not as ugly as a full, color-agnostic bloom effect (which some really-cheesy-looking games do). I was puzzled by how that could possibly work. Well, the key word is was :D

I discovered, quite by accident, that compositing a bloom effect onto the scene using a geometric average produces a remarkably-nice effect. By geometric average I mean it's a multiplicative blend: sqrt(a * b). The result is really interesting. Not only do dark colors spread slightly (producing a lovely hazy effect that's perfect for when you're inside a dust cloud), but...it's almost as though it provides a bit of a fake global illumination effect. I was really surprised and amazed by how much this tiny little hack improved the look of the game (well, at least in my opinion...some of you may feel that it bestows too much of a dreamy / soft quality, and I'm sure I will provide the option to turn it off or lower the intensity)

I played around with this for hours, tweaking the exact compositing formulae and messing with the blur pass. In the end I think I've managed to produce a really lovely effect. After thinking about it for a while, I guess I've come to somewhat of an understanding of why it looks so good. The thing about a geometric average is that it preserves dark detail more than an additive composite. Geometric averages are influenced heavily by near-zero values. Think about, for example, the fact that in a geometric average, any element going to zero causes the whole average to go to zero (meaning a single element can dictate the value of the entire average, no matter how many terms it contains!) - quite a different effect from an arithmetic average, where more and more terms cause each term to become "less and less" relevant. Practically speaking, this means that geometrically compositing the bloom does a great job of preserving the "detail" of the original scene. Dark details on a texture remain dark. But, it does modify the low-frequency color in a way that...well, spreads color across the screen (it's a blur). So in the end, you have the high-frequency detail of the original scene, but you allow the low-frequency detail to bleed into neighboring regions (via the blur)...and the result...feels like a subtle global illumination. Very cool. And very cheap! :geek:

After that excitement, I went on to discover a new, exciting formula for visualizing the background in dusty areas of space! Previously I was rendering a blurred version of the environment map. I really dislike how it kills all of the beautiful detail in the nebulae. On the other hand, having high-frequency detail at long distances in a dusty area looks wrong, because the light should scatter and high-frequency detail should be lost. But today, I discovered that simply reducing the contrast of the high-frequency environment map looks acceptably-convincing, but still gives you all that beautiful detail to look at! Totally unrealistic, but hey, if I can have my cake and eat it too... :D

I went on to improve the dithering and tonemapping algorithms that come at the very end of the render chain. I know I've said that banding is due mostly to video compression, but there was still a slight amount visible in game. But that's been totally blown away with today's improvements in dithering...so now there's absolutely no color banding, even in dark & low-contrast areas. Looks wonderful :)

Whew. Always feels good to get those little graphics bursts out of my system...and always leaves the game looking that much better :thumbup:

[ You can visit devtime.ltheory.com for more detailed information on today's work. ]
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: Week of October 13, 2013

#4
Wednesday, October 16, 2013

Summary

The nice thing about good graphics days is...they have this lovely tendency of coming in pairs :D Today was about one thing and one thing only: antialiasing.

I've mentioned before that I'm frustrated with LT's AA implementation, which is called FXAA. It works well for removing jaggies, but since it's just a rather blind post-process effect, it also works well for removing a lot of crisp texture detail from the scene. So much so, in fact, that I've been playing with it turned off as of late. I didn't even have it enabled for the last dev video, which explains why you might have seen jaggies.

So today I set out on a quest. I reviewed all the modern AA techniques. I downloaded the code for SMAA, which is supposedly the state-of-the-art new amazing thing (CryEngine uses it). But after looking at the code for a while (which was, to be fair, a lot less ugly than the atrocity that is FXAA), I just couldn't stop thinking...aliasing is such a simple problem. Surely we can solve it with a simple post technique. Surely we don't need these three passes and hundreds upon hundreds of lines of messy code. And you know what? I think I was right :geek:

Ten hours later I'm pleased to present...RSAA!!! :D "Really Simple Antialiasing" :lol: Also "Really Stupid Antialiasing," depending on how you want to look at it. RSAA is one pass and 80 lines of code. It's faster than FXAA but looks better (at least in LT). It's depth-based so it preserves texture detail but eliminates geometry-induced jaggies. It's configurable not via a thousand #defines, but via a single integer that controls how many blur pixels it can extend a jaggy by. I've set it to 7 right now...meaning it can take a jaggy and extend it across 7 pixels if need be. If you're wondering, that equates to super-high-quality AA. At that level of quality, geometry-based aliasing is...totally and completely blown away. Like. 100%. The reconstruction doesn't always look 100% perfect, but it tends to look really, really good. I'll keep improving it I'm sure, but for a first attempt, it's already..good enough to ship :)

So...mission accomplished! Yet another concept that proves to yield a simple solution under the right insights :D I'm so pleased at how much of a difference this makes. No jaggies but still high texture detail...ahh....so good!! :geek: And I'm so happy that I now have a "really simple" AA technique that I can just drop in to any future projects :monkey:

Between this and yesterday's new bloom effect, I can't get over how good the game is looking! Of course, that's totally subjective but...there's one thing that isn't subjective: it looks way, way, way better than it did during the Kickstarter ;) It also looks way better than I ever expected it would, to be quite honest. Just goes to show you...all it takes is...endless hours of work :D

:squirrel: The AA squirrel likes his crisp details. :squirrel:

[ You can visit devtime.ltheory.com for more detailed information on today's work. ]
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: Week of October 13, 2013

#5
Thursday, October 17, 2013

Summary

Well, the truth is...not a great day. Research is a bigger villain than I expected. :monkey:

Luckily, I still came through with a good bit of productive work, but I'd be lying if I said it was anything more than a distraction from finishing research. I wrote quite a bit of code today for meta-c++ code generation. Programs to write programs, you know :ugeek: The reason is...writing the code that backs my reflection macros is very tedious. I have to write a macro that works for 1-parameter functions, one that works for 2-parameter functions, and so on. They're all very similar but not similar enough to actually express using some kind of shared construct. Really the only good way to express the patterns is to write a program that writes the code ;) Today I finally solved the issue once and for all by writing a program that generates the header file. Now I can effortlessly generate code that supports...however many parameters I want! :D But the important thing is that I didn't just write a program to do so, I also built a lot of tools and bits of code to support metaprogramming, so now I have a little toolkit to pull out whenever I need to do something like that again. Heck, you might even call it a "procedural program" toolkit, since it's generating code :lol:

It's not just cool because it saves me time and energy, it's also cool because it's one of my first steps into a larger world. I've talked about metaprogramming before, but never from this angle (the angle of actually writing a program to write a program). At this level, there's literally nothing that you can't do, and the syntax can be as nice as you want it to be. I'm sure this isn't the last time that LT will benefit from this kind of technology :)

Anyway. About the real task at hand...I'm really choked up on the finer details of research, and the brain juice just isn't flowing for some reason. The exact mechanics of how a technology progresses as you gain levels in it, how technologies can be combined, and how a technology unlocks a constructable type. All the same thing, but still...I just don't have the clarity. I mean, I understand the basic answer to each of those. But the exact details and interactions between them...still don't have the clarity. Where is it!!?!?! :shock:

I'll get there. I really will :shifty: Just...not as quickly as I was hoping :think:

[ You can visit devtime.ltheory.com for more detailed information on today's work. ]
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: Week of October 13, 2013

#6
Friday, October 18, 2013

Summary
A full day of laying siege to the great Castle of Research Mechanic :squirrel:

Thanks to another epic Gazz thread, I had a fresh perspective on research mechanics...and it also helped that I didn't get distracted with metaprogramming today :oops: Between some fresh ideas in that thread and continued development of old ones...I think I've got the answer! ;)

What I'm trying to do now is build a mock-up tech-tree-viewing widget so that I can get a real sense for how research is going to feel.

But alas, building this widget presents it's own problems :think: For too long I've avoided custom 3D render widgets that aren't related to the game world. I've got a great architecture for supporting arbitrary views into the world (command interface, ship viewer, etc), but they assume...well, that there's a world. For something like a tech tree, I want a render view into a "fake world," if you will, where I display some cool 3D effects and such to make it look like a techy tech tree, rather than my usual boring 2D widgets that you see everywhere else. So I'm working on cleaning up and generalizing some pieces of the rendering code to allow this kind of thing :)

Really, I just want to play with the research mechanic "in person" before I decide on it. I need to see the tech tree, see how it plays out with the operations that I plan to support, etc. Try-before-you-buy kind of thing, you know? Then I can decide whether the current design works well or not :thumbup:

[ You can visit devtime.ltheory.com for more detailed information on today's work. ]
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: Week of October 13, 2013

#7
Saturday, October 19, 2013

Summary

Ok, look, I failed you guys. I didn't get research done. In fact, I didn't even touch research today.

Because I was too busy playing with massive planets and systems :D :D :D :D :D :D

For whatever reason, this morning I decided that I wanted a break from research mechanics and wanted to go ahead and move forward with the double-precision upgrade. The next phase of the upgrade is changing the engine so that all rendering is done with the camera centered at the origin. Well, I was honestly surprised at how quickly that change came and went. But that was only the first of many wonderful surprises today :) It took remarkably little code to shift all of the coordinates on the CPU before sending them to the GPU. Even more surprisingly, it took remarkably little tweaking for everything to appear correctly as it did before! What a stroke of luck (I thought this part was going to be really painful!!) :D

But. That was just the beginning of the luck ;)

As I flew around in my new, camera-centered world, I noticed an undeniable smoothness. It was smooth before, but as you flew further and further out, you could quickly start to feel the FP errors. But as I flew around this new world, I could feel no jittering, even when I ventured quite far outside of the central system area!! So, feeling that my luck was sure to come to a close sooner than later, but still retaining a tall drink of that Newbie Optimism (tm), I decided to scale things up a bit. 10x planet size, 10x system size.

And the engine handled it like a champ :ugeek:

Just like that, I was torn out of the cutesy, playful world that we've all come to know as Limit Theory.

A massive planet loomed in the distance, warning me that this new place - this vast, cold swath of space - was a place of grandeur, a place of greatness. Great scale, great peril, great potential, and great reward. Yes. Here, I sensed, would great things transpire. The vastness of this space demands nothing less. Here, in this suddenly-larger sandbox, ambitions will take hold of men. The invention of life-stealing weapons in the name of science. The production of war machines in the name of industry. The discovery of strange anomolies, the ensuing exploitation of information for profit. Friends becoming allies, allies becoming armies, and armies becoming graveyards. The accumulation of great power, and the cruel and untimely robbery of that power by the hand of fate. Factions rising and falling with the tides of countless alien worlds. Exploration, expansion, exploitation, and extermination. Yes. We will see it all here.

I like this new place. You guys were right...scale really does make a difference :)

(I actually don't fully understand why this change fixed our precision problems, but it appears that we won't be needing a double-precision engine after all :D )

[ You can visit devtime.ltheory.com for more detailed information on today's work. ]
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: Week of October 13, 2013

#8
Week Summary

General Sentiment

On the one hand, I remained fairly halted this week at the conceptual impasse of research mechanics. On the other hand, with an awesome new bloom effect, a silky-smooth new AA technique, and much bigger system scales, it was definitely a killer week for graphics :D The visuals are better than I ever imagined they would be. I mean, it's still not CryEngine. It never will be...but I'm proud of our little LT..it's come a long way :)

Now, since I've already peered into the future (aka wrote this summary later than I should have :oops: )...I have a really good "feeling" that the research mechanic impasse will be lifted early next week ;) :roll: In fact, you might say I have a good feeling that I'm on the edge of a breakthrough :D

Major Accomplishments
  • Implemented function reflection in preparation for ever-more-powerful UDF / modding capabilities
  • Implemented a really nice new multiplicative bloom effect, resulting in a serious visual boost!
  • Implemented a new antialiasing effect, resulting in another serious visual boost
  • Implemented camera-centric rendering; increased the in-game scales by an order of magnitude :cool:
  • Significantly reduced mouse latency, making LT feel more responsive
  • Started on a new metaprogramming toolkit for writing programs that write programs


Last bumped by JoshParnell on Mon Oct 21, 2013 10:08 pm.
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford

Online Now

Users browsing this forum: No registered users and 4 guests

cron