Sunday, December 8, 2013

Another day of massive slackery

Just like yesterday, I let my graphics self go to town during the first half of the day. Today's fun improvements: moving / dynamic dust (continuing the work from a few days ago on the new dust effect, but now with motion), and that classic scifi effect of tiny space particles that stretch along the direction of motion. The former looks great and adds a subtle feeling of life even when sitting still in a nebula, as you can see dust clouds subtly changing form over time. The latter is a surprisingly-impactful visual cue that conveys the feeling of motion. If you watch carefully you can see little bits of space dust streaking by when moving at high speeds. Though subtle, the feeling definitely helps lend a sense of scale to the scene, as well as a sense of significance to motion. Love it With the dust and space particles in good shape, I'm left with only one lingering graphics task inspired by X Rebirth: more lights.

And AI? Oh. AI. That thing. I killed it today. As I write this dev log, I'm watching my 16 miners strip a system of ore on the other screen. 16 NPCs, full reasoning, 4x time acceleration, 100 FPS. Before optimization. So far they've collectively exchanged 41,000 units of Rochabite at the station It's been running for 30 minutes or so and no issues. Yayyy!!! And still very intelligent selection of nearby rocks. Awesome

I basically ripped out the entire creativity mechanism today. The previously-mentioned "neural" PDF sampling was a good idea...in theory. Like every good piece of theory, it crashed and burned in practice. The sampling mechanism was causing NPCs to become too locked-in to ideas. Even at low retention rates, the neurons would quickly escalate to having a few elements being 100x more prominent than the rest. I ended up pushing the decay rate so high to achieve "out-of-the-box" thinking that it effectively defeated the point of sampling from a PDF in the first place. Good-bye neurons. Hello extremely-simple ring-buffer. Trivial to code, everything is constant-time and contiguous in memory. Works way better than all my other "fancy" ideas...how typical Simplicity is taunting me again

Thanks to today's and yesterday's improvements, the AI has literally never been so fast and accurate. I was able to perform a 128-person test today for the first time. 128 people planning out their lives and executing intelligent actions in one system! Now that's what LT's all about! I've got a ways to go before I can hit 60FPS with that population, but most of the slowdown is the fault of the game engine and entity logic, not the AI! Awesome

Getting more confident in the AI every day

Ah, and also a fair bit of work on grid and list layouts for the nodal interface today. But who has time to discuss such mundane things when one has graphics and AI to talk about

PS ~ Apologies for strange dev log hours...I'm nocturnal again at the moment

Monday, December 9, 2013

I. Am. Exhausted Highly triumphant. But exhausted

Last night I made an executive decision: today would be a full-blown Graphics Josh day The goal would be the final sprint of my three-pronged X Rebirth-inspired graphics improvements campaign: a better dynamic lighting solution. It would require an uprooting of several areas of the rendering architecture in order to move to a so-called "deferred" renderer. Hence giving him the whole day. But he pulled through

Very pleased that the monkey was able to get that whole thing done in a day No small task, to be sure. I also took the opportunity to do a sweeping cleanup of the LT rendering architecture. Less code, less GPU memory usage, higher performance, that sort of thing I feel significantly more comfortable with it now.

Visually, the lighting looks the same (as it should), minus the nasty tiling artifacts from the previous technique It does run a fair bit faster as well. Once I start adding more lights to ships and stations, we should really start to see a difference I'll also probably experiment with "headlights" at some point this month, since people have asked several times about that feature and it would be easy with the new rendering pipeline

After some serious sleep, I'll be happy to get back to the non-graphics work, and will probably lock that kid up for a while now since he's already done a lot over the past week

Tuesday, December 10, 2013

Hmmm...not the day I wanted it to be...I guess I was due for an off-day

I started out with the intention of focusing on something to which I haven't given much attention lately: sound. This week I finalized a contract with the sound designers who'll be doing the sound effects (not music) (I'll introduce them in the next update ), so it's inspired me to give a little more thought to the sonic experience of the game and implement some more sound-related infrastructure. I was particularly keen on exploring the idea of creating procedural system ambiances by blending ambiance loops in various ways. All this recent graphics work has got the systems looking so beautiful and atmospheric, I feel like they deserve some equally-rich sounds to accompany them.

Sadly, I got carried away, as I often do when going back to old systems. A lot of the sound-related code is poorly-designed, probably because it's the first time I've ever written that kind of thing, and it's always hard to get the architecture right the first time Accomplished lots of refactoring, but not much tangible progress.

To be perfectly honest, I'm not entirely sure where the day went, which is strange, because I've been here sitting at my desk for all but two meals of it Must be caught in a temporal rift!! Now if only I could figure out how to get it to go in the opposite direction

Well...I'm going to go ahead and get a head start on tomorrow, because I think it needs to be a more impressive day

Wednesday, December 11, 2013

Here's how my day started off: I laid down on the carpet, put earplugs in, and assumed fetal position for about six hours, thinking about nothing but AI. Yep. Always a good start to the day

But really. I'm getting tired of not having the right solution to AI, so it's getting pretty intense over here. I'm pushing harder and harder. The perfect solution to life, the universe, and everything still evades me....even though I spent a tremendous amount of time and sacrificed a generous number of neurons on the altar of AI today.

Sure, I made plenty of breakthroughs. Using explicit time valuation instead of per-time value to improve the long-term thinking of the AI (no longer using value per unit time, but rather, total value including time value). I made lots of other improvements as well, which resulted in new all-time records for how many credits the NPCs were raking in per unit time. I imagine they were pretty close to optimal in this particular situation

But that wasn't the focus. Delegation. Parallelism. I need them. NOW. I came within two inches of ripping out multitasking and replacing it with a new system. Today I have started to view parallelism in a different and more powerful way: as a scheduling problem. Consider that every actor in the universe has a certain number of independent "resources" which can be scheduled to achieve goals. For example, you have your ship, scanners, production units, etc. These are all independently-schedulable resources. You can schedule a production queue on your prod unit, you can scan enemies off the port bough with your scanner, and you can move to a station in hopes of acquiring some crazy new weaponry with your ship. All of these resources can be scheduled independently. If you're a faction leader and have subordinates, those can be treated just like any other resource. Like I said, I was about two inches away from rewriting the core algorithm again to respect the nature of independent scheduling.

But I decided to maintain the current architecture. I like how the current architecture enables serial and parallel actions to be described in the same way. Today I implemented serial actions, which are little more than a compound of multiple actions. But, the result is profound! My AI is a fundamentally-backwards AI No, not as in it marries its cousin. As in, it starts at a goal and thinks backwards from that goal. But there are some tasks that require a forward solution (a "plan"). One such task is accepting a contract or mission. To accept a contract, you need to have a plan to solve it, otherwise you don't know how much it'll cost you to solve, which means you have no way of knowing whether the contract is a good deal or not. Today I introduced forward-solving to the AI, in the context of the overall backwards-solving algorithm, by introducing a serial action, which is the counterpart of the parallel action. I quite like that duality

In order to really make this work, I am going to need a heuristic suggestion function for finding relevant actions. I can't ignore it anymore. The truth is, when you start dealing with extended plans (serial actions), and parallel actions, the probability of finding a good solution drops exponentially. If you are to command >10 units, for example, there's simply no way that you'll find a good scheduling of those 10 units if you're doing a random search through a 10-ply parallel action. You need a heuristic that gives sensible options.

Apart from the heuristic, I am in good shape. Then again, AI seems to be a remarkably-asymptotic problem. Every day, I write my log, telling you about how far I've come with the AI algorithm. And I'm not lying!!!!! Yet, still, every day, there remains an ocean to cross. Oh well. As I've said before, this is one aspect of the game on which I won't compromise. Rain or shine, I will build this AI. And it will understand parallelism, delegation, and the finer nuances of reasoning.

Thursday, December 12, 2013

Listen. I'm not in the habit of fabricating dev logs. So I'm going to come clean.

He got out.

Please don't blame me. I was tired and my neurons were fried from yesterday's massive AI-conceptual-gauntlet-extravaganza. I must have forgotten to lock the cage. I must have forgotten. But that's ok. I won't apologize for loving what I do If you're wondering whether we're past the point of diminishing returns when it comes to letting Graphics Josh work, let me go ahead and answer that in the clearest and most direct way possible: NO. Absolutely, 100% no. The thing I love about graphics Josh, unlike AI Josh, is that each and every time he comes out, the game gets significantly better. Or shinier. Whatever. No such luck with AI Josh

To be fair, today wasn't just a graphics day. It was also an optimization day. My framerate had dropped to about 40 FPS on my desktop at full settings. Not acceptable! I wanted to get that back up to 60. I did Here's what happened today:

• Much faster lens flare rendering using lower-resolution buffer and faster math
• Much faster deferred lights using screen-space quads instead of light volumes, + faster shader math
• Better tonemapping for a more balanced, rich color to the scene
• Better starfields with more interesting variations and more stars (looks beautiful!!!)
• New, physically-based metal shader using Cook-Torrance (huge improvement!!!)
• More work on balancing the nebula generator to produce consistently-beautiful systems
• Integrated planet shading with deferred renderer, planets now look great

The Cook-Torrance shader is really a tremendous improvement over the previous metal. The dynamic lighting has never looked so lovely!

Honestly, I think you guys are going to love the December update. LT has come so far in this month alone. It's really growing into the beautiful, atmospheric experience that I always wanted from Freelancer 2 AI and gameplay are on the way, I promise!!!

As many of you probably know, the Elite: Dangerous alpha has just been released yesterday. Watching the videos, it looks incredibly impressive (especially compared to what was shown at the time of the Kickstarter). I'll probably buy it (with my own money, don't worry ) soon, but it's already inspired me to get my act together. I won't sleep tonight. I will overclock my brain to 250%. It's time time to push hard. Really, really hard.

As my genius CS140 professor used to say, "a little slope makes up for a lot of y-intercept"

(Cookie for you if you understand the meaning )

PS ~ I want to say sorry to those who frequent the forums for my disappearance as of late. I'm really trying to push as hard as possible this month, so I'm not spending much time at all on the internet in general, including the LT forums. I still love you all, but I need to make this game happen

Friday, December 13, 2013

One more day. Give me one more day. I will have it.

I've been wrong enough times. But that's fine. It's coming to an end. I'm finally starting to understand it. I started with one algorithm, I moved to another, and what I realize now is that they're both pieces of the real thing. I've been missing the big picture, beating around the bush. Forgive me. But I see it now. It's the exact same algorithm that we use to render photo-realistic images efficiently. Bidirectional pathtracing. I should have seen it all along. At first I came at it from planning. Chaining, backward search. Then I came at it from gradient optimization. Forward descent. With back-propagation. Then I realized I needed forward chains too. My God. I looked at it from so many different directions without realizing what the computer was really trying to tell me.

But that's ok. I see it now. Delegation, reactivity, parallelism.

Please, just one more day

Saturday, December 14, 2013

Look guys. I messed up numerous times with AI. I'm not going to deny it; that's just the truth. But you hired an Indie dev, that's what you got The good news is: I learned from it and I moved on

We are now at a level of AI that the LT engine has simply never seen before. It's a unification of the previous planning AI and the recent gradient AI. Combined, they make one heck of a team. To be completely honest, I think I've just re-invented the STRIPS planning algorithm, combined with bidirectional search. It's all been done before. But that's OK! I came to it with my own understanding. I understand this AI. That's what's important. The fact that it has all been done before in the research papers is just a comforting re-affirmation that what I'm doing is worthwhile

The beauty is that it can plan in all directions. Evaluating actions based on their immediate value? Yes. But propagating value from far-off goals in order to construct a long-term goal? Absolutely. This is it. The AI I've been looking for. It evaded me for so long. But I won't despair nor will I apologize. LT was built to be an incredible single-player experience, and I will do whatever it takes to ensure that such is the case. Contracts are on the way!!! Parallelism is on the way!!!

It's not done yet, but it's already displaying the foresight that the previous algorithm never had, coupled with the creativity that the first algorithm couldn't achieve. Maybe I'm a perfectionist, and maybe you'll accuse me of too much change. But, as for my own personal reasoning, I place an infinite value on technologies that will prove superior in the future. Maybe that's my flaw; but it's also what made me go for the LT dream. This is going to be big.

Parallel and delegation fit in seamlessly with the new AI architecture, it's just a matter of hammering out a few kinks. This one is final, I swear We are so close.

Week Summary

(Written 1 week later...)

Truly a ridiculous week. This is how I wish every month would start Graphics, AI, heck, I think there may have even been some sound thrown in there. The metal is shiny, the planning AI is deep, the sound is crisp...it's all coming together.

Let's keep powering through December

Accomplishments

• Switched (back) to deferred rendering, major optimizations to lighting pipeline...can now support lots of dynamic lighting!
• Implemented dynamic, moving dust clouds
• Major improvements to nebula generator
• Major optimizations to lens flares
• Completely new, physically-based metal shader
• Major improvements to star fields
• Evolved the sophistication of the AI one step further, moving to a bidirectional planning AI