Return to “[Archived] Daily Dev Logs, 2012 - 2015”

Post

Week of May 19, 2013

#1
Sunday, May 19, 2013

Summary

Alright, now it's starting to look like planets are just as dangerous as nebulae in terms of things that will suck up your time :think: Really need to move on but....it's just so fun trying to perfect them!

Today I primarily worked on introducing randomization to the new planet generator - exposing all of the new controllable parameters and finding good ways to randomize them. I'm pretty happy at this point with the amount of variety in the planets, although we still need a few more "types" of planets. Gas giants are the main thing that's missing, but I feel like that could easily consume another day or two, since I haven't even started playing with them yet :?

I started the universe generator today, but didn't get very far - I feel like I'm still missing some intuition / clarity. I know how to make everything work, but it's just...not as clear as it should be. Maybe I just need to stop being a wimp and dive in :|

Bad hours today because I treated myself to some quality BSG time this morning, as a reward for the good work on planets as of late. Seasons 2 & 3 are just so gripping I can hardly tear myself away... :oops: I'll do better tomorrow!

Hour Tally

Coding: 3.88
Internet: 3.19
Testing: 1.39
Total Logged Time: 8.47
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: Week of May 19, 2013

#2
Monday, May 20, 2013

Summary

Didn't get sucked into planets all day! :clap:

I actually moved on, to...nebulae!!! ....wait a minute. I feel like....like I've been here before... :think:

No but really, I promise I'm not going in circles. I needed to expand the background generator to include randomization (like I did with the planets yesterday). So now we've got new randomized backdrops + new randomized planets, and the universe is really looking great!! I also worked a bit more on the lighting of the nebulae and improved the scattering to yield more natural / soft color variations.

Now...the part that we all know I've been avoiding...universe generation. I made some progress today, despite still feeling that I'm lacking in clarity. I have the code to hierarchically build clusters / regions / systems, but no connectivity yet. Connectivity is really the biggest problem. Again, I have a solution, as I made sure that I could fulfill on the promise of an "infinite universe" before I launched the KS (I finished the algorithm just a few weeks before launching the campaign, actually!) But I keep thinking that maybe I can find some simplifications. The simpler the structure of the universe, the less work it will take to generate and simulate it. I also keep calling into question whether it was really the right decision to promise an infinite universe. On the one hand, a real universe should be so large that, from a computational standpoint, you need to consider it to be infinite, because you cannot possibly enumerate and simulate every entity within. On the other hand, at that scale, I wonder if it starts to lose meaning? Realistically, how many systems will a player ever touch in a single game? Would it actually increase the coherence / importance of systems if you knew there was a boundary? Very difficult questions to answer, primarily because they are psychological in nature.

At any rate, even though I'm sure I'll do a whole lot of whining about the pain of simulating such a large universe in the future, in the end, I know it will be an important opportunity to develop some really good simulation code. Better to solve the really big, interesting problem "well" than to solve the small, easy one "perfectly"...right? :eh: I guess that's my philosophy with this game, I mean, promising procedural & infinite everything...a rather large problem :lol:

Hour Tally

Coding: 6.60
Internet: 2.59
Testing: 3.32
Thinking: 1.40
Total Logged Time: 13.91
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: Week of May 19, 2013

#3
Tuesday, May 21, 2013

Summary

Didn't plan it, but today turned out to be all about the ships 8-)

To really nail procedural ships, it's going to take many more days like this. Not because I think procedural ships are particularly hard, but because I'm no artist/modeler, and I don't even know how to make a ship by hand, much less with formulas. Still, I'm going to cling to that belief that all things are simple until I can't cling any longer (and by that time, hopefully I'll have found the secret to ships). I did make some good progress today, and developed a few new tricks and insights that will help me in the end. I still don't have a good structural understanding of ships, but I'm getting there. I am realizing more and more the importance of distorting the domain. A few, well-placed global distortions can turn an ugly mess of cubes into a sleek, contoured hull. Moreover, distortions can create very pleasing patterns that give the impression of more complexity than actually exists. For example, parallel lines under a continuous, nonlinear distortion will still roughly follow one another. This can mean, for example, that the outline of an inner shape flows smoothly with that of the outer shape, creating the illusion of intentional design. Very interesting stuff. I just need to collect a whole toolbox full of these insights, and then hand them over to the computer.

On top of working with the geometry, though, I continued to improve the plating texture as well as the lighting model. The plating is getting better all the time, but I still haven't found the right fractal. This one looks too unintentional, too sloppy. More like random squares than anything with a strong pattern. This is one of those fine lines that I am finding it very hard to walk with fractal algorithms - it is very difficult to break the visible patterns, even more difficult to break the patterns "only to an extent," so as to retain some degree of coherence. As I've said before, it's a tough beast to tame...but I think the payout will be immense. As for the lighting model, it's been improving since it was born so many months ago, and I'm happier and happier each day with the look of the metal. Today I have finally solved the issue of ships being too dark in dark systems. Figuring out how to make ships look good in dark systems has been a major problem since day one (and probably all too apparent during the early days, when you could hardly see ships at all), but it's over now :thumbup:

Finally, I'm continuing to wrap my head around universe generation via quality thinking time. I'm nearly ready to proceed, but am waiting on one last piece of insight: dynamic resource loading / unloading. Once a clean architecture for this finds its way into my head, I think I'll be ready to really tackle this beastly problem!

PS ~ I finally broke my cycle of nocturnalness today, and it felt great to see some daylight while working!! I think not ever seeing daylight was depressing me a bit :shifty:

Hour Tally

Coding: 5.30
Internet: 2.64
Testing: 2.95
Thinking: 2.19
Total Logged Time: 13.08
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: Week of May 19, 2013

#4
Wednesday, May 22, 2013

Summary

Despite lackluster hours, it was an awesome day for ideas! :clap:

Yesterday I mentioned being hung up on resource management. It was the last thing holding me back from really pushing ahead on universe generation. Today, I had a very welcome rush of ideas concerning dynamic resource management! It was one of those glorious moments where everything just hit me all at once, and all the complexities vanished. I also realized that I could unify a lot of different systems with this scheme. I have been beating around the bush with special-cased code like collision mesh generating and caching, audio file loading and streaming, lazy loading of shaders, etc...but now I understand that they are all the same thing!

The insight, in its most basic form, is that one must think of a resource as being both an algorithm for generating a piece of data, and the (potentially nonexistent) result of that algorithm. Whether or not the result is actually resident in memory is something that we really shouldn't even have to worry about! Let someone else worry about it. Define a resource by the algorithm that creates it, but don't actually create it until needed. Let whomever needs to use it create it. Let a scheduler determine if and when the actual resource should be loaded / evicted. Let someone else do the work! As long as you provide the algorithm, the resource is well-defined! When you create a star system, you will not need to load textures or models or anything. You will simply attach the appropriate algorithms, and then if someone wants to render the star system, they will use those algorithms to generate the necessary resources (if the resources haven't already been generated). It's so obvious, so simple, yet so elegant :thumbup: Should have seen it a long time ago. It is, fittingly, a very nice reflection on the procedural mentality: think of content in terms of what generates it, not in terms of the content itself.

But there's so much cool stuff that goes along with this idea. For example, you can think of caching a resource in a file as being an "algorithm adapter"! Define caching to be the following algorithm: if cache present, return contents of cache, else run child algorithm, save result to cache, and return contents. This generalizes the notion of caching so that it requires no extra work to provide a cache layer on top of any generating algorithm. For example, instead of all the specialized code that handled loading and generating the system backgrounds, now I only have a few lines that define the actual algorithm for generating the backgrounds, and then a single line that wraps the algorithm in a cache adapter, which ensures that the backgrounds will be saved to / loaded from a cache file instead of generated each time at startup. The caching algorithm is generalized using the serialization engine, so it can support any data type. You could just as easily use it for caching the state of an AI player, a system, even the whole universe. And all of it happens automatically, without writing any file operation or serialization code!

There are so many cool possibilities for this system...I'm so excited!!! And I've only scratched the surface :D

I spent most of the day refining and implementing this idea. I've got a lot of the code in already, but I'm going to have to go through the engine and replace a lot of old code to use this new idea. In the end it will be a beautiful simplification that will simultaneously provide a tremendous amount of new power.

Ah...smell that? It's the sweet smell of code being ripped out and replaced with cleaner code :)

Hour Tally

Coding: 4.98
Internet: 2.88
Testing: 1.59
Thinking: 1.45
Total Logged Time: 10.90
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: Week of May 19, 2013

#5
Thursday, May 23, 2013

Summary

Now, I've had my fair share of hardware problems. I've had RAM give out on me, I've had sassy GPUs get overheated and angry with me for pushing them so hard. But, before today, I had never had a machine get so angry with me that it actually committed suicide right before my eyes. Well, now I get to add that to my repertoire!

Seriously, I thought the whole dynamic resource system idea from yesterday was pretty great, and figured my laptop would be as excited as I was to hear about it. Since it hasn't gotten much love lately, I decided to spend the day coding on it instead of the desktop. All was well, and together we were having a positively joyous time ripping out ugly code and injecting the crisp flavor of elegance into the engine. Or at least, I thought we were having a joyous time...until I smelled the burning :shock: Apparently my laptop was actually so opposed to the idea of this resource management system that it decided it would take no part in the implementation. So without the slightest warning, it did what any reasonable machine would do, and promptly Harakiried itself right there on the spot. When I try to turn it on now, the screen is black and the side fan emits a prominent burning smell. You could have just told me you didn't like the new system, I would have listened :cry:

So, that's how my day went :problem: I moved back to the desktop and re-coded all that had been lost. My desktop, thankfully, seems to agree with the changes, as it has yet to grace me with any alarming odors! Sadly this little stunt by my suicidal laptop cost me a 50% efficiency loss for the day, since I had literally just finished overhauling the engine when it decided to end its life.

Still, technological issues aside, I'm pleased with how my new resource system is working out so far! It's mostly implemented now, and it looks every bit as clean as I imagined. Most resource generation is now deferred / lazy-loaded (and will be subjected to fancy scheduling algorithms at a later time). In theory, I should now be ready to construct regions without resource issues! Hopefully I'll find out tomorrow :)

...assuming my desktop doesn't get angry at me :shifty:

Hour Tally

Coding: 5.93
Internet: 4.12
Testing: 0.19
Total Logged Time: 10.24
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: Week of May 19, 2013

#6
Friday, May 24, 2013

Summary

I guess I got pretty backed up on code-ripping during the weeks leading up to LTP, since I couldn't afford to make major changes. Now that we're passed that, though, the rush of urges to tear things apart and cleanse them is hitting me full-force, as it did yesterday, and as it continued to do today.

From time to time, I take a moment to ask myself what I would do differently if I could start over from scratch. Thanks to my profound love for destroying things and replacing them with shinier things, the answer is usually fairly short. I'm very happy with a lot of the engine's architecture. Perhaps the most consistent answer that comes to mind, however, is the way that I structured the scene graph (the structure that keeps track of all of the game objects). I opted for a flat graph - all objects within a "context" (context being a container of sorts: a system, the interior of a space station, the surface of a planet, etc.) are treated as equals. It was one of the first pieces of code I had to write, because it's so central to the engine. And I know now that it was the wrong choice. I should have gone with a hierarchical scene graph. At the time, my reasoning was that the conceptual clarity of having every object on the same level would win out in the end. But I have come to understand that both the simulation and rendering of the scene would be more elegant under a hierarchical solution. For example, currently, a weapon that's mounted to a ship is considered to be an object just like any other within a system. Naturally, there is a special relationship between the ship and the weapon that ensures that the weapon stays "attached," but they're still considered to be objects within the system. A better way would be for the weapon to be considered "in" the ship, and not in the system. The ship would be in the system, and the weapon in the ship. It would solve problems with far more clarity. I'm sure this is how most engines do it (hence the "graph" part of scene graph...), but hey, I was bound to make a few rookie mistakes, especially in the very beginning when I barely knew what 2+2 was :angel:

Anyway, the reason I bring it up is that the issue has been bothering me even more than usual lately. As I prepare to transition from local to global simulation, I am finding more and more that the flat scene graph is very unnatural when you start to consider the broader scope - like the region. In a hierarchical scene graph, a system would have the exact same relation to a region that a weapon has to its ship. Elegant. But with a flat scene graph, everything is a "special case." A system is the primary container, but at the same time is contained in a special way by a region, a weapon is contained in a system but has a special logical relationship to the ship to which it is attached, etc. Not elegant. If you follow the logs, you know that the phrase "not elegant" is pretty much a death sentence for any piece of the engine :roll: After it's been deemed less-than-optimally-elegant, it's only a matter of time before the executioner comes in.

Unfortunately, this particular thing is rooted so deeply within the engine that execution will not be as clean as usual. I will need to carefully prepare my team, study the layout of the target area, know where the shadows and security cameras are. We will need heavy weaponry but light feet. It will not be an operation for the faint of heart :silent: Today I began planning out this operation. I've made a bunch of changes already to clean up the way the scene works. It's just a preliminary step in this operation, but now that I have started down this path, it is only a matter of time before we strike :ugeek:

In slightly-less-technical news, I started implementing the jump gate graphics today, and ended up doing some pretty neat stuff with it. Looking good. I'll bet you can guess what kind of math I used :)

Hour Tally

Coding: 5.80
Internet: 2.06
Logistics: 0.58
Testing: 0.44
Thinking: 2.57
Total Logged Time: 11.44
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: Week of May 19, 2013

#7
Saturday, May 25, 2013

Summary

Finally got to start on some serious universe generation today!

For the first time, I implemented a simple regional star map so that I could see the results of the generator. After a few hours of playing with the generating algorithms, I was very pleased with the output! The simple algorithm that I came up with so long ago turns out to do a nice job of creating variety but also structure. It produces compelling arrangements of systems that I think will be fun to explore :thumbup:

As I've said before, there are going to be a lot of improvements that will need to be made to the engine in order to step up to handling an entire universe. One of those improvements is memory handling. Some of the code surrounding object generation is a little too loose with memory, and I'll pay the price for that in the long run. So part of what I did today was to start combating this. I set up a very simple loop that creates a piece of a universe (~1000 systems), then deletes it immediately, and does it all over again. The result is the creation and deletion of thousands of little universes per second, and hundreds of thousands of systems (...millions of asteroids :geek: ). The nice thing about this is that if there is any memory looseness, then this little procedure of burning through universes is going to find it immediately. This program will quickly consume your entire RAM unless memory is handled perfectly. So I used this to hammer out as many memory issues as I could find, and in the end, was proud to end up with stability! Thousands of universes burning before my eyes without any change in memory usage. Yay :clap: Naturally, this is only the beginning. When you start bringing in AI and factions and such, things are just going to get infinitely more complicated. But we'll tackle it one step at a time!

Also continued to improve nebulae and stars today. The nebula generator is really solid, but it's going to take some more time to make it consistent. Balancing consistency and variety is the biggest challenge with any procedural algorithm, as they are at odds with each other. It's easy to make an algorithm that is consistently good but has no variety. Likewise, it's easy to make something with a lot of variety that will have no consistency (rand()rand()rand() 8-) ). I would say that the nebula generator could be better in both departments right now, so I just need to have patience and keep fine-tuning it until release.

Hour Tally

Coding: 4.78
Internet: 2.94
Logistics: 1.25
Testing: 2.34
Thinking: 1.12
Total Logged Time: 12.42
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: Week of May 19, 2013

#8
Week Summary

General Sentiment

I'm slightly disappointed with this week. I can see the hours and the improvements, but I can't help feeling that it could have been a bigger week. In general, May could have been a bigger month. Not sure what's going on, but I think this week at least it had something to do with the fact that I'm struggling to return to "normal" sleep hours, and my mind has been a bit dazed as a result of the dramatic shift. Really need to get back into high gear...hopefully June will be the beginning of "the long sprint"!

Major Accomplishments
  • Implemented fancy new dynamic resource system
  • Started on universe generating algorithm
  • Started on new ship generator
  • Improved planets, nebulae, and stars
  • Improved internal memory handling in several places (at the expense of thousands of innocent universes)
  • Improved scene graph infrastructure in anticipation of "operation hierarchical scene graph" ( :shh: )
Combined Hour Tally

Coding: 37.28
Internet: 20.41
Logistics: 1.83
Testing: 12.21
Thinking: 8.73
Total Logged Time: 80.45
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford

Online Now

Users browsing this forum: No registered users and 3 guests

cron