Return to “Dev Logs”

Post

[Adam] Friday, May 11, 2018

#1
Update time! Let's jump straight in.

What have I been working on lately? A little bit of everything, as usual.
  • A boatload of UI polish and new features.
  • A cleanup pass on the engine.
  • Completely overhauled the way we generate engine bindings for Lua.
  • Reorganized and simplified our Lua support code.
  • Implemented the 'control bar' for switching between e.g. ship control, command view, etc.
  • Refactored camera control to allow smooth transitions between different cameras.
  • Re-implemented the command view.
  • Designed zone control mechanics.


UI Polish
UI elements now store their local position instead of their global position. The global position thing was sort of an experiment to see what it actually ends up looking like in practice. It certainly has a few pros. It's dead simple for one. Comparing positions and checking for intersection is simple. It doesn't make it much harder to support different resolutions, as one might initially expect. On the other hand, once you have something like scroll views it gets a little hairy.

My first thought was to have the scroll view modify the view matrix at the renderer level. This way child elements of the scroll view would never even know they were offset. This was nice since dealing with an offset didn't leak out of the scroll view itself, but it caused a performance hit on some machines due to an OpenGL quirk. Storing global space also meant parent elements would have to pass a delta position to all children when the parent moved. And adding children with a relative offset from the parent was trickier since sometimes we build chunks of UI before attaching them to the UI and therefore without knowing their global position.

Storing local position and origin simplifies all of that. Sure, it means we have to think about whether we're want to be in local or global space, but it ends up largely being pushed down into helper functions and we have have to do that for 3D objects anyway. It actually ended up reducing the amount of code in a few places in the UI elements themselves.

At Josh's request I also did some light refactoring of the inheritance model of UI Widgets. I wasn't happy with the inheritance to begin with and taking the time to stare at it as a whole and contemplate the pros and cons has utterly convinced me that inheritance is the wrong way to share code.


Lua Binding Generation
We run a script when compiling the engine that parses header files and outputs a set of Lua scripts that the game is able to load so it knows how to talk to the engine. Previously this was...less than ideal. The tool produced type information, but we had to manually write the bindings for each API. We had to manually define and flatten some structs. We had to annotate headers the tool wasn't able to parse correctly. Commented out code was parsed. LT specific helper functions couldn't be defined alongside the API functions.

Before the Global Game Jam Josh wrote a replacement parsing tool that was much simpler, yet more powerful. We used it at the jam and I liked they way it worked, but it was only 50% complete. Luckily, this time around it's in Lua instead of Python where I'm much more comfortable, so as one of my 'fun day' tasks I decided to finish the tool and migrate over to using it. And oh boy did it pay dividends. This tool handles everything.

We're able to automatically convert our C style engine interface into idiomatic Lua object code. The engine types are defined as opaque structs and every function that starts with 'TypeName_' is added to a metatable for the type. Functions that take a pointer to the type become object methods and the reset become 'static functions'. 'TypeName_ToString' functions are automatically bound to __tostring metamethods, which means print(engineType) just works. Structs visible to Lua are parsed, flattened, and sorted to put dependencies first. Commented code is ignored, preprocessor checks are evaluated, and warnings are emitted when preprocessor checks exist that may not match.

Function pointer typedefs are parsed. Enums with underscores are split into hierarchical tables. 'Metadata' is stored so other code can enumerate all engine types. Currently this is useful for creating CType entries for native engine types. The tool outputs a single 'loader file' that loads the engine DLL (taking into account 32/64 bit and debug/release configurations), and a binding file for each engine API. The whole thing returns a table hierarchy that can be used like so: PHX.TypeName.APIFunction(). And there are hook points defined so that, when loading a set of API bindings, the game can inject additional functions into a 'namespace' and have them be indistinguishable from true engine API. Previously we had quite a few 'helper scripts' which contained functions the game needed but didn't quite belong in the engine. Trying to remember if Lerp is in PHX.Math or Math is...dumb.

So what does this end up looking like? Well, here's the original C header
Spoiler:      SHOW
Image
And the generated bindings
Spoiler:      SHOW
Image
Note how Directory_Close and Directory_GetNext have been mapped to object methods close and getNext while everything else was mapped to non-method functions. onDef_Directory and onDef_Directory_t are the hooks for extensions. Here's what those extensions look like
Spoiler:      SHOW
Image
We don't ever have to think about bindings now. This tool is awesome.


Engine Cleanup
After fixing up the bindings I was reminded of, and annoyed by, just how haphazardly scripts were organized and loaded. We had Limit Theory scripts, Phoenix scripts, and general Lua utilities just clumped inside LT. Our other tools and testbeds always end up reimplementing the same general utilities because they aren't easily reused. I separated it all into 3 layers: Env, PHX, and LT and moved the first 2 into our shared assets folder. Env is general Lua utilities that don't rely on the engine, while PHX is engine bindings, extensions, and utility scripts that depend on the engine.

I also standardized a bunch of the Env scripts, added helpful functionality, and fleshed out unfinished ideas. My favorite products of that are requireAll and Namespaces. requireAll is a straightforward way to load all scripts in a directory recursively and return a hierarchical table. Under the hood it's using the built in require and package.path which means it works completely seamlessly alongside idiomatic Lua. Namespaces let us inject and optionally flatten those tables into the Lua global symbol table. No prefixing a bunch of code with PHX or Env. PHX.Vec3f(0, 1, 0) gets simplified to Vec3f(0, 1, 0). But the PHX table still exists for disambiguating symbols when necessary. Previously we had manually written scripts that loaded every script in a directory (non-recursively) and returned a table. I especially enjoyed nuking those.

There was also a ton of smaller stuff involved like standardizing header layouts, macro name casing, simplifying ArrayList, tackling some old TODOs, separating LT and the 'launcher' code.

One of my favorites was updating the Lua stacktrace that is printed during a crash. It already printed the names of all functions on the stack, but now it prints local variables, function parameters, and upvalues. It uses any engine provided ToString functions or Lua provided __tostring metamethods for friendlier printing. And it highlights any nils using ANSI escape codes. Together, this means 9 times out of 10 we instantly know exactly what went wrong, rather than having to spend a couple minutes scanning the code for issues or trying to reproduce the crash. Seeing as Lua is awful and lets you crash at runtime because of mistyped variable name, this happens quite often and the extra output already saves us a ton of time.

These backlog, cleanup type tasks can be a nice way to relax after more difficult work. The reward-to-effort ratio is huge.

But...you're sick of infrastructure stuff, right?


Command Interface
Getting back to gameplay, I started working on re-implementing the command interface. I started by codifying the concept of a Control. From an earlier post you may recall that the simulation is an autonomous thing and the UI simply allows the player to poke the state of the simulation. Controls are the UI panels that accept player input and do the poking. There's a Control for each method of interaction with the simulation. For example, the ShipControl when piloting, the CommandControl when commanding a fleet, or the DebugControl that lets us view and edit internal machinery. Only one Control is active at a time, but a single Control can contain arbitrarily complex UI within it.

The first step toward implementing that was to add a MasterControl that determines which Controls are available and lets you switch between them. This is visible as a small bar at the top off the screen where you can change the active control, very similar to what was in the prototype. It auto-hides and has shortcut keys and all that jazz.

Switching out an active tree of widgets exposed a couple issues in the UI system. For this to work smoothly I added the ability to enable and disable widgets. Structurally this is a smooth transition that can happen with a fade or other animation. Previously we'd just destroy and recreate widgets as necessary because it's cheap, and honestly we could have continued doing that, but it ended up being cleaner to enable and disable as needed. This way Controls can maintain state when inactive instead of having to stash that information somewhere and re-load it next time.

I also reworked the way widgets are added to and removed from the hierarchy. We defer adds and removes so we don't have to worry about the list of widgets changing while we're in the middle of iterating though and updating them. Previously we processed adds and removes at the very end of the frame. That wasn't ideal for a few reasons. 1) We'd draw a removed widget for one more frame after it was removed. 2) We'd not draw an added widget until the next frame. 3) The first time a widget was updated it would not have a valid layout. This all stems from the order in which UI events are processed:

Code: Select all

Input
Update
Layout
Draw
By moving the add/remove logic from after draw to between update and layout we fix all 3 of those issues. I also added an extra mouse focus check after add/remove so there should never be any form of one frame delay on widgets appearing/disappearing, gaining/losing focus, extra/skipped updates, etc.

Next up was making sure switching between camera types was smooth. The ship control uses a 'chase camera' that follows close behind the ship. The command control uses an 'orbit camera' that can be freely rotated and moved. These camera types are actually just movement logic. We have a 'real camera' that handles the viewport and updating the rendering matrices. I modified the cameras to write position and rotation as the final output so it's simple to calculate an offset and lerp it to zero when switching cameras, which gives a perfectly smooth transition. This should have been extremely straightforward, but it turns out our rotation math is not consistent across all parts of the engine. I spent more time than I would have liked digging through our quaternions and matrices to understand what was going on. I didn't end up completely fixing it because it's tricky to do without breaking existing code and I didn't want to spend the time on it right then. I did write fixed versions of the broken code and added some tests to make it easier to suss out other issues when the time comes. This is a good candidate for my next 'fun day' task.

On the visual side I wanted to add the 'holographic view' of the previous command interface. I dug out the old holographic shader and implemented the ability to globally override rendering.

Then, of course, I had to get the meat of the control in: unit selection, setting and restoring unit groups, and issuing orders. Selection works in the obvious way: click and drag to select, hold ctrl to add to selection, shift to remove from selection, or both to invert selection. Since ships have this habit of moving around constantly I added a button to focus on the current selection. It moves the camera to the center of the objects and zooms to fit them on screen (taking into account their bounding boxes). And for fun I added a way to lock focus so the camera will follow selected objects when they move. It's quite satisfying to select your allies, order them to attack some poor miner, and sit back and watch it play out. It feels almost theatrical with the camera smoothly following the action.

Of course this all lead to more UI iteration. I ensured keyboard focus moves appropriately when dealing with menus appearing and disappearing. I added 'modal' windows that are automatically closed/cancelled when you interact with something behind them. I improved the way containers calculate their size during layout passes so things like context menus get clamped to the screen automatically. I combined the old 'refresh focus when widgets are added/removed' and the new 'refresh focus when widgets are enabled/disabled' and drastically simplified it.

Here it is in action. Note that the visuals are all placeholder, this hasn't had a beautification pass.
Spoiler:      SHOW

Design and Next Task
Now that we're solidly in gameplay I'm going to need to do occasional design work to help Josh flesh out some systems. To that end I did an initial design of how zone control is going to work. Josh then ordered me to play some Freelancer to ensure I understand the heritage of Limit Theory.

Next up on my list is docking mechanics. The first pass will be the infrastructure: keybindings for docking, knowing when it's possible to dock, swapping out the current control with a docking control (merchants, storage locker, etc), and changing to some fancier camera. The second pass will be iterating on that until it feels nice. And a third pass will add some transitions and generally just make it sexy.

Phew. That's a bit of a wall of text. I'll try to make the next one shorter.

P.S. Tess has gotten pretty big!
Spoiler:      SHOW
Image
Post

Re: [Adam] Friday, May 11, 2018

#2
Nice update, Adam, thank you!

I don't think it's that we're sick of infrastructure as much as we're wary of diving down the rabbit hole again and getting consumed by development of the tool rather than the game: it's the all-in focus that we've seen before and don't feel is appropriate at this stage.

It's fantastic to see you working on the actual game - that little video is beautiful to watch and really encouraging!
--
Mind The Gap
Post

Re: [Adam] Friday, May 11, 2018

#5
:clap: :clap:
Ringu wrote:
Fri May 11, 2018 12:35 pm
Nice update, Adam, thank you!

I don't think it's that we're sick of infrastructure as much as we're wary of diving down the rabbit hole again and getting consumed by development of the tool rather than the game: it's the all-in focus that we've seen before and don't feel is appropriate at this stage.

It's fantastic to see you working on the actual game - that little video is beautiful to watch and really encouraging!

The interesting thing about the dynamic that Adam and I have going on is that I'm really good at finishing 90% of the work what I perceive to be 90% of the work but is actually more like 50% of the work :ghost: Adam, on the other hand, is the kind of detail-oriented person who can drive home the last 400% of the work (listen, we have some advanced math going on here in the office, don't bother trying to keep up :V).

So what Adam calls a "fun" day, others might call a "clean up after Josh's spree of finishing-the-first-halves-of-things" day... :ghost: Anyway, my point is, when Adam is doing infrastructure work 1) it's usually my fault, and 2) it's always relevant to finishing a feature (I wish I could say the same about me...)

Of course, now that he's played some Freelancer, his life will be getting harder since we can saddle him with both finish-the-infrastructure work AND gameplay design work ;) :lol:

In all seriousness, great log, great work. His commits vastly outnumber mine this month :thumbup:
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: [Adam] Friday, May 11, 2018

#6
JoshParnell wrote:
Fri May 11, 2018 12:51 pm
In all seriousness, great log, great work. His commits vastly outnumber mine this month :thumbup:
To be fair, Josh, yours were mostly in giant chunks, whereas Adam made many smaller ones. You put in more work than you're letting on. :P Adam's done awesome work, though, for sure. :thumbup:
Have a question? Send me a PM! || I have a Patreon page up for REKT now! || People talking in IRC over the past two hours: Image
Image
Image
Post

Re: [Adam] Friday, May 11, 2018

#7
Outstanding all the way around -- great teamwork, and excellent focusing of Adam's time.

I enthusiastically welcome Adam rationalizing the "guts" of LT's myriad systems because I know that's going to pay off in faster post-release support for fixes and enhancements. Basically, this is "productionizing" LT... and that's a marker of a professional-grade development plan.

Naturally, I'm also stoked to hear about gameplay features in the works. :D

I'm most happy to hear about thinking on zones. There's a phenomenal amount of gameplay fun that can emerge in a gameworld where overlapping volumes of space can have different functional textures.

It's also satisfying to see that actual implementation of fleet control features is happening. Even if my first love will always be the 4X/strategic planning level, I know a lot of people are looking forward to the "RTS" level, and I'm happy to see progress there. Love that video clip!

And I'm curious: what core design concepts are still TBD? I get playing Freelancer to appreciate its look-and-feel, but what parts of Limit Theory still need concepting? (I'm interested in this mostly from a game design perspective, but it's also true that this could raise questions about feature creep in the minds of some.)

Finally, I'm still waiting for word of any specific progress on what I believe are certain major gameplay milestones:

  • dynamic generation of new star systems during play
  • competent generalized task breakdown and delegation by NPCs at multiple levels (tactical, operational, strategic)
  • LOD AI that's reasonable, performant, and fun operating across hundreds of star systems
  • all key features implemented (even if simply) and being balanced for whole-game fun

These aren't complaints -- just me looking forward to future updates.

Thanks!
Post

Re: [Adam] Friday, May 11, 2018

#9
JoshParnell wrote:
Fri May 11, 2018 12:51 pm
The interesting thing about the dynamic that Adam and I have going on
Heh. It does sound like the perfect partnership, dividing the work up to best effect.
I honestly believe that the moment you recruited Adam was the moment LT the game really started to take off.
listen, we have some advanced math going on here in the office, don't bother trying to keep up :V
You're the only developer I've ever known to use quaternions and calculus in project estimates :-D
So what Adam calls a "fun" day, others might call a "clean up after Josh's spree of finishing-the-first-halves-of-things" day... :ghost:
I have to be honest here, that's my sort of fun day too :-)
I'm always at my best when I'm optimising the living daylights out of some existing code; @Adam, you're a rare breed and I doff my cap to you, sir.
Anyway, my point is, when Adam is doing infrastructure work 1) it's usually my fault, and 2) it's always relevant to finishing a feature (I wish I could say the same about me...)
Ahh, I hope you're not feeling that I'm complaining about doing the infrastructure work: my intent with that remark was actually to reassure both Adam and you that doing that work is *not* a bad thing, and we do enjoy hearing about it still; for me, it was only the work on an item that was secondary to the game and engine and was kinda beginning to take over, that I felt was in danger of being all-consuming, that I was (what's the word? disappointed? Even that seems a bit too strong - maybe 'leery' is right) leery of.

Guys, I think you're making great progress now, and it's great to read about the cool things you're working on - for me, this is exactly what I wanted when I backed the campaign and it's fantastic to see.
Of course, now that he's played some Freelancer
I'm beyond thrilled that you've broken his Freelancer cherry, to the point that I had to break out the emulator and have a bit of a fly around myself at the mention of it :-)
In all seriousness, great log, great work.
To you too, Josh, you too!
--
Mind The Gap
Post

Re: [Adam] Friday, May 11, 2018

#10
AdamByrd wrote:
Fri May 11, 2018 11:46 am
Josh then ordered me to play some Freelancer to ensure I understand the heritage of Limit Theory.
Best boss ever!
AdamByrd wrote:
Fri May 11, 2018 11:46 am
Phew. That's a bit of a wall of text. I'll try to make the next one shorter.
Don’t you dare. Thanks for the exciting mix of behind the scene and visible gameplay progress.

You and Josh seem to make solid progress these days.
:thumbup:
Image
Post

Re: [Adam] Friday, May 11, 2018

#14
AdamByrd wrote:
Fri May 11, 2018 11:46 am
Only one Control is active at a time.
So i cant have some big local map view (with command and control) open on one screen and my personal flight controls on my main screen?
Or just my inventory controls open in some corner while flying?

Or is it always "you are doing this one exact thing right now!"?
AdamByrd wrote:
Fri May 11, 2018 11:46 am
To that end I did an initial design of how zone control is going to work.
And no info on that?
let us theorycraft!
Post

Re: [Adam] Friday, May 11, 2018

#15
Cornflakes_91 wrote:
Sat May 12, 2018 3:42 am
AdamByrd wrote:
Fri May 11, 2018 11:46 am
Only one Control is active at a time.
So i cant have some big local map view (with command and control) open on one screen and my personal flight controls on my main screen?
Or just my inventory controls open in some corner while flying?

Or is it always "you are doing this one exact thing right now!"?

That is a great question. I also wondered about this.

Cornflakes_91 wrote:
Sat May 12, 2018 3:42 am
AdamByrd wrote:
Fri May 11, 2018 11:46 am
To that end I did an initial design of how zone control is going to work.
And no info on that?
let us theorycraft!

What Cornflakes said! :D

We actually had a great conversation about zones a while back in this thread, based on Josh's comments in his March 2014 devlog.

I'll exercise my dark necromantic powers over that thread and see if there's more to say there now.

Online Now

Users browsing this forum: No registered users and 4 guests

cron