Week of September 14, 2014

View Daily/Weekly Statistics about LT Development

Week of September 14, 2014

Postby JoshParnell » Mon Sep 15, 2014 1:01 pm

Sunday, September 14, 2014

Tremendous :)

LT App Launcher

I mentioned a few weeks ago the idea of having a single executable that simply runs an LTSL 'application' script -- a script that describes the full functionality of a program. Today I've finally seen the birth of that tool! My primary motivation behind this is to be able to maintain a bunch of different small LTSL applications that I can use to test various pieces of the game in isolation. Previously I had a few hard-coded variants of that idea: the full-blown Limit Theory (which I would often tweak in certain ways to test specific facets of the game), a single-system testbed that simulated one system and presented a top-down, UI-only map for me to poke and prod as the simulation ran (I developed most of the dynamic economy in this application), and even a console-only version of LT that simulated a universe at maximum time compression (200-300x real-time).

With a unified LTSL application launcher, I can now do all of this and more by simply writing new LTSL scripts that use specific pieces of the game. I can and will use the launcher to build simple test applications for things like benchmarking engine performance, prototyping and measuring the performance of new AI algorithms, quickly building individual UI widgets, and more! In time, Limit Theory itself will become a script that opens the main menu, builds universes, etc. I've not quite got enough exposed to LTSL yet to make that leap, but it's actually not far off at all :) I don't think I need to point out how much benefit this launcher idea has -- but just to drive the point home, I'll also say that it will allow me to always keep a working version of LT, since I won't have to screw with the real game anymore to test things. Come monthly update time, we should, in theory, be seeing far less "gotta fix that!" moments once the playable LT is cemented into an LTSL script.

Currently, I'm already using the launcher to create new UI widgets, as well as to create benchmarks for LTSL performance (in fact, I was very surprised by the results of my first few benchmarks -- LTSL executes significantly faster than I would have guessed! Even in its fairly-unoptimized state!)

AI Testbed

With the advent of the launcher, one thing that I'm looking to do very soon is set up a tool for benchmarking AI algorithms. In order to develop quality AI (dogfighting, trading, piloting, management, etc.) I'll need to have precise ways to measure the results. By setting up and executing simple test scenarios in LTSL, I envision a world in which I can actually reduce the efficacy of an AI algorithm to a single number! In the case of dogfighting, for example, I will create a very simple (perhaps empty) system, spawn two identical ships, load a 'baseline' dogfighting algorithm into one pilot (for example, the simplistic one that they're all using right now), then load the algorithm to be tested into the second pilot. With a nearly-empty system, I'll be able to run a full-detail dogfight at a time compression factor of 100+ -- meaning I'll know who wins almost instantly. I can run 10, 20, perhaps even 100 rounds of the same dogfight, and output a final win percentage and average damage taken for the tested algorithm. Talk about easy-to-understand feedback!

Over time, I anticipate being able to continually build up algorithms of increasing skill until they're ultimately at a point where they blow away the old one. At that time, they'll become the new baseline for the next test, and I'll keep moving on up!

Breeding AI Algorithms... :twisted:

That's great, right? But is it as far as we can push the idea? Surely not...:geek: Testbeds get us automatic quality feedback. That's one piece of the puzzle. What's the other piece? Automatic exploration of the solution space. If you marry an automatic solution grader with an automatic solution generator, something magical happens: a feedback loop of self-improvement that requires no intervention. It's no coincidence that evolutionary computation has become an increasingly-popular and powerful tool. It provides a way to automatically explore a space of solutions. Again, when that exploration is automatically guided by a function that can determine the quality of a solution, ....magic :ugeek:

What I'm getting at is this: if we can develop a mechanism for generating random AI programs, then all we need to do is plug that mechanism into the AI testbed and wait for our AI to rule the world.

I've got a lot more ideas surrounding this one, including true 'skill' levels for AI pilots by using algorithms of different AI testbed performance ratings. I've also got some fun ideas about how this will ultimately form itself into a 'ranking' system in the game, allowing players (AI and human alike) to work their way up a tiered skill ladder in different areas of gameplay by comparing their performance to one another. Perhaps you'd like to become one of the few feared 'Black Diamond'-level dogfighters in the universe? 8-)

---

Fun times ahead :)

PS ~ First time ever successfully pushing back the infamous devlog deficit? Just more proof that there is time ;)
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
User avatar
JoshParnell
Developer
 
Posts: 4261
Joined: Sun Oct 07, 2012 3:06 pm
Location: Baton Rouge, LA

Re: Week of September 14, 2014

Postby JoshParnell » Tue Sep 16, 2014 1:38 pm

Monday, September 15, 2014

Excellent :)

Of all of today's work, I'd have to say that my appreciation lies most with the realization that I had today that led me to a much better model of widget updating / drawing, and may, in time, do the same for game object updating / drawing. For a long time, I've had some kind of a fuzzy idea about the relationship between an object's state, the way in which an object interacts with its children to determine that state, and the way in which an object yields control of its state to the parent.

Interestingly, my fuzziness has manifested in a lot of mistakes and imprecisions over time -- from the early days of the LTP interface, which has some strange parent/child consistency issues, to the days of the flat scenegraph, which mostly circumvented and ignored the structured relationship between objects. Finally, I believe I've come to an understanding of the proper two-phase relationship that makes updating and drawing work very consistently for hierarchies. As of today, this understanding is fully-implemented in the UI and already showing a lot of new clarity! Yay :)

I won't delve too far into the details, because there's been a lot of that lately -- but just explain the basics. When an object (or this case, a widget) has some properties (like size, position, color, etc.) that may be determined by children, but may also be overridden by parents, the way to handle it is in a two-phase update process. First, we traverse the hierarchy of objects in what I call the 'pre' way -- we call the pre-update on our children so that they can figure out their state. We then determine our own state, using only ourself and child configurations. For example, a list widget may determine what its total size needs to be in order to hold all list elements. Then, we traverse the hierarchy again in what I call the 'post' way -- we read from our own state (knowing that parents may have altered it), and write to the state of our children, before finally allowing them to perform their post-update. In the pre-update phase, a listbox, for example, will compute the total size necessary to hold children (children have already run their pre-update to determine their desired state). In the post-update, a listbox, for example, will recognize that the size that it now has is the 'global truth' (determined by parents), and will position and size children accordingly, before allowing them to do the same recursively. It's a bit like every object in the hierarchy first listens to what his subordinates are doing, then figures out what to do, then reports back to and awaits orders from a superior. The state first travels up the hierarchy. After receiving those orders, he creates his own sub-orders for children and dispatches them. The state now travels down the hierarchy.

Anyway. That might have been a bit more wordy than I was hoping for, but it conveys the general idea :) Using this technique, I am able to get exactly what I've always wanted: the ability to freely make decisions about local state, as well as for those decisions to be hierarchically considered (and modified) by the 'higher' pieces of the hierarchy. State becomes both local and global, but without any duplication or intermediaries. It all just works (finally!)

I keep getting the urge to over-extend with all of this newfound tech power. I keep wanting to make the mistake, as I did when August was drawing to a close, of simply trying to 'make it all happen.' But I mustn't. Not yet. Not before the time is right. I really do think my post two days ago was on to something. To ride the exponential, we can't overextend in either direction (content or tech) before the time is right, or we'll fall off into the world of linear progress (perhaps even sub-linear?) Small content, small tech changes. Climbing our way to the top, but carefully. That's the idea of September. We still have two weeks left :)

PS ~ Weekly updates? Yeah, let's drop that. Not sure about you guys, but I always felt them to be of rather low utility. Minor daily dev logs, major monthly reports. I like that style. Sub-minor weekly summaries? No, I don't think they really suit us.

PPS ~ Hoping to push that devlog deficit back further tomorrow :thumbup:
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
User avatar
JoshParnell
Developer
 
Posts: 4261
Joined: Sun Oct 07, 2012 3:06 pm
Location: Baton Rouge, LA

Re: Week of September 14, 2014

Postby JoshParnell » Wed Sep 17, 2014 1:39 pm

Tuesday, September 16, 2014

Work work work :)

Despite continued work on all the little applet children of this new 'app launcher' revolution, I've not had any grand moments or terribly interesting thoughts enter my head today. Just pushing forward, piece by piece. Still almost two full weeks -- love that.

I'm bursting with anticipation to get my AI testbed up-and-running. I didn't yet have time, but I'm planning on bringing it online tomorrow :) Doing so will be an awesome step forward -- although I don't think it'll take much time, it will force me to bring all that I need to bring into LTSL for getting universes set up / launching scripted game scenarios. From there, the possibilities are going to escape into the infinite :)

See you soon :wave:
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
User avatar
JoshParnell
Developer
 
Posts: 4261
Joined: Sun Oct 07, 2012 3:06 pm
Location: Baton Rouge, LA

Re: Week of September 14, 2014

Postby JoshParnell » Thu Sep 18, 2014 4:30 pm

Wednesday, September 17, 2014

Yikes! How does that devlog time creep up on me like that?? :shock: Sneaky...

Today, unfortunately, I decided to bite the bullet and sacrifice a bit of would-be AI fun time in order to dig in and straighten out some issues that have been bothering me for a while. A lot of small things, like improving the way that many engine objects are managed (via reference counting instead of autopointers), simplifying the thread and threaded job management, and, most importantly, significantly simplifying the scalar field model implementation.

Scalar field models have been a pillar of LT's procedural model generation since that fateful day long ago when I rediscovered my love for the elegance of distance fields. Unfortunately, the actual implementation has always been a bit of a hairy beast. There's really a lot going on in there. When you see an asteroid in LT, it has actually already put both your CPU and GPU through quite a few hurdles to make it onto the screen :) First, there's the computation of the actual scalar field data. The LT Engine automatically generates GPU shaders that compute the value of a given field. It runs those shaders iteratively to produce a 3D texture, and then ships that off to a second thread, where the texture data is put through the well-known Marching Cubes algorithm to produce a mesh. This all happens several times for varying levels of detail. Finally, there's a bit of a hairy ambient occlusion calculation that takes place, again on the GPU. Only then is the mesh ready to render.

That's all well and good, but the complexity of it scares me (you know I love my simplicity), and it's also adding too much to the game's start time. It takes about 5 seconds to load a system on my top-end machine. I know, that's not much, but we can do better ;) Most of that time is spent jumping through the scalar field algorithms.

Today, after a fair bit of pain (complexity always does induce that...), I was able to significantly simplify all of the code. It took a lot of bug-hunting before I was able to nail it back into place; dealing with code that runs on multiple CPU threads PLUS multiple GPU shaders is always a pleasure :shock: But it's done! Everything is the same as before, but the simplicity is finally apparent :) I will now be able to simplify this all very quickly, and hope to make some breakthroughs on the speed of those algorithms soon.

Anyway. Today was a bit of a sidebar, but again, it addressed a very specific problem that is a rate-limiter for me at the moment. That's how to do it! Tomorrow: AI testbed! Seriously!

:)

PS ~ Interesting side-note about complexity. Although the feature set of LT has been continually expanding over time, especially in the past few months with LTSL, the actual codebase is now shrinking. Through my aggressive search for simplicity, I'm constantly able to shrink the code while still maintaining the same speed and functionality. Truly, someday LT may be a few thousand lines :lol: Love it! :monkey:
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
User avatar
JoshParnell
Developer
 
Posts: 4261
Joined: Sun Oct 07, 2012 3:06 pm
Location: Baton Rouge, LA

Re: Week of September 14, 2014

Postby JoshParnell » Fri Sep 19, 2014 3:41 pm

Thursday, September 18, 2014

Got it :)

I'll admit, it was by the hair of my chinny chin chin, which is why I don't sound more enthusiastic about it...yet :cool: What I've pulled off as of today is not all that impressive, to be honest, but it's yet another first step forward in isolating the elements of the game so that we can augment and scrutinize them carefully.

As of today, we've got an AI sandbox LTSL app creating a basic system, spawning two ships with identical load-outs, tossing a pilot into each, and then pitting them against each other until death. The whole scenario plays out without any rendering of the actual system, which means the only CPU cycles we spend are on letting two obviously-angry guys blow one another to pieces. My only real metric right now is the score count, which I draw in a basic UI (again, set up with LTSL). I also draw two health bars to watch exactly how a match is going. None of it is particularly interesting yet, since the UI is minimally-functional and the actual results aren't telling us much (2 identical ships with identical AI scripts...yeah, as you might guess, the win rate converges on 50%).

Still, I've got a deep excitement in my bones for this work. The first of many AI death matches has taken place today. The first of many. :ghost: :ghost: :ghost:

On another front (a front which actually took 60% or so of my time today), we've got a pretty huge update for LTSL brewing. For quite some time, I have had the creeping feeling that 'String Tree' was a slightly incorrect way to think about the ideal syntax structure for compilation. Guess what I now believe to be optimal? Surprise, surprise...'String Lists'! Even more (less?) surprising, they're basically exactly what LISP uses (string trees were slightly different). LTSL is being converted over to string list format -- as of today, the parser is already written and working :) Syntactically, this does not change any of the existing LTSL (well, function declarations can look slightly prettier with string lists). But it does open the door for more power. A lot more power.

I don't want to dig too deep today, as this work is only just starting and I'm sure you'll hear more about it when the conversion is finished...but infix binary operators (3 + 5 * 10 instead of (+ 3 (* 5 10))) and easier parametric types (var myArray (Array MyType)) are just a few of the applications. It was actually the latter that finally convinced me to start the conversion. To properly do compile-time parametric types in LTSL, I really needed some syntax that string trees won't permit. More power + cleaner / faster syntax = happy Josh :monkey:

More to come ;)
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
User avatar
JoshParnell
Developer
 
Posts: 4261
Joined: Sun Oct 07, 2012 3:06 pm
Location: Baton Rouge, LA

Re: Week of September 14, 2014

Postby JoshParnell » Sat Sep 20, 2014 2:30 pm

Friday, September 19, 2014

Wow :)

Still maintaining full-speed-ahead on UI and AI testbed fronts, but today's show must be stolen by something far more exciting (to me): the completion of yesterday's LTSL work.

It wasn't terribly difficult, nor did it eat much of my time today -- but the payoff was and is unreal. The LTSL compiler and parser are now using 'string lists' rather than 'string trees' to do the dirty work, as per yesterday's log. What may seem like a triviality at first glance has just unlocked a big old pot of elegance :geek: After I did the conversion and got everything working again, I had to take this new beauty for a test drive. So I embarked on what might seem like a pretty large task: implementing infix operator notation with correct precedence (e.g., making something like 2 + 5 * 6 / 8 + 9 evaluate correctly). Indeed, in the regular world of parsing massive languages with ambiguous grammars and so forth, it might not be such a simple thing. In the simple world of LTSL string lists, it took me one hour to have full-blown infix operators with correct precedence implemented in the language. The code that implements it is about 80 lines of C++. Simplicity pays off once again! :monkey:

Practically-speaking, what this means is that my script-writing life has just gotten a lot easier. The prefix notation was one of the few things really contributing to the rate-limitation of writing logic in LTSL. This:
Code: Select all
if (< josh.GetDietCokeLevel 1.0)
  josh.AddItem dietCoke (+ 1.0 (/ 2.0 josh.GetStomachCapacity))

Was getting old fast. Now it's almost too easy:

Code: Select all
if (josh.GetDietCokeLevel < 1.0)
  josh.AddItem dietCoke (1.0 + josh.GetStomachCapacity / 2.0)

Perfect :)

She's come a long way, our little LTSL. I couldn't be more proud of where it stands today -- I'm not sure there's any language that I'd prefer to write code in. Regardless, I can say beyond any shadow of a doubt that I would take LTSL over C++ all day any day. Granted, we're comparing apples and oranges, since C++ brings a lot of extra technical power to bear. But are we? Does it? I don't need templates and polymorphism in LTSL to write the LT game logic, of that I am certain. But some day, far beyond the fall of LT 1.0, I have a feeling that, much like infix operators, the other 'complex' features of languages can be unraveled into simplicity in this little language that I have learned to call my home.

September has been a fantastic month so far. Really. And with ten days still left on the clock, the sky remains the only limit :D :squirrel:

PS ~ I know, you're tired of hearing about LTSL and would rather hear about big spaceships going boom. Too bad! My devlog! Mine!!! :roll: :monkey:
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
User avatar
JoshParnell
Developer
 
Posts: 4261
Joined: Sun Oct 07, 2012 3:06 pm
Location: Baton Rouge, LA

Re: Week of September 14, 2014

Postby JoshParnell » Sun Sep 21, 2014 3:59 pm

Saturday, September 20, 2014

A joy. A pure, unadulterated joy. That's the best I can do at describing what it feels like to finally have the exact toolset that I want, and to use that toolset for hour after hour of lighting the screen up with anything I can dream of. I've finally reached the point where there's no friction between the mind and the machine :)

Anyway. Frankly there's nothing conceptually interesting to discuss today, but that doesn't mean it wasn't an awesome day! UI, AI, and an hour-long 'fun break' of random loading screen prototyping (because playing with LTSL + graphics code + instant reloading is absurdly fun and productive). It's amazing what an hour and 80 lines of LTSL can do (I know, I know, I should really start putting screenshots in the logs...sorry... :roll: ).

Hard to believe I've still got 9 more days of this ahead of me. Feel like the Limit Theory singularity is rapidly approaching :shock:

:wave:
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
User avatar
JoshParnell
Developer
 
Posts: 4261
Joined: Sun Oct 07, 2012 3:06 pm
Location: Baton Rouge, LA


Return to [Archived] Daily Dev Logs, 2012 - 2015



Who is online

Users browsing this forum: No registered users and 6 guests