[Adam] Friday, July 20, 2018
Posted: Fri Jul 20, 2018 12:50 pm
Friends. Compatriots. Limit Theoreticians. I come bearing a dev log.
As always, I've been bouncing around like a madman. Sprinkling a little feature dust here, vacuuming a few bugs there, reinforcing scaffolding, ensuring the house stays tidy as we scale, etc etc. Oh, and I may have finished an entire engine system along the way. First up, Docking!
Docking
Last time I showed off the command interface. A large part of that was ensuring we can switch between 'contexts' smoothly. UI gets swapped out, the camera animates, bindings are changed out, and so on. These are small things that are going to be leveraged frequently. After finishing the command interface I wanted to continue working on gameplay and also push on these features a little more to see how they hold up. Docking seemed like a good fit.
We wanted to start with just the core mechanics: fly close to a space station, press a button, auto-pilot to the docking port, and see UI menus for everything you can do at that station.
First, a dockable component. The UI simply searches for dockables and if you're in range presents a button prompt to begin docking. The act of docking actually removes the player's ship from the world and adds it to the station instead. Similarly, once docked an undock prompt is presented.
I extended the MasterControl I implemented last time to add 'control sets'. The UI looks at the player each frame to determine which control set ought to be active and automatically switches to it. Previously we could choose from piloting, commanding, and debug controls. Now, when the player is docked we swap to a control set with things like your storage locker, merchants, and the jobs board. Since we can already swap out UI trees easily this ended up being dead simple to implement.
With all the state control in place, I wanted to make it a smooth, physical docking operation where you literally fly into the station. The AI is implemented through an action stack. Actions are pushed onto the stack and AI simply run their current action each frame, popping them once complete. Conveniently, the player is no different. The action stack is just empty and the UI controls poke the player's state directly. (This actually surprised me when I built the command interface because I could select my own ship along with my allies when giving orders and my own ship would fly along, taking part like any other unit.) I added a new docking action that is essentially the 'move to location' action that also happens to reparent the player from the system to the station once at the destination. And boom, with a whopping 7 lines of code you now auto-pilot right on in.
If a station is destroyed a fraction of the damage is inflicted on docked entities and they're released from the station. If your ship survives the damage, great, you can limp away. If not, well, your attacker gets to rummage through the debris of your ship along with the station.
It's not a particularly big or complicated feature, but I'm thoroughly pleased with how brain-dead easy it was to implement. The way Josh structured AI and actions is excellent. There's a ton of power and flexibility there with no real complexity. The whole of implementing docking took a day, with about 3 hours being the 'real work' and the rest polish and iteration.
Physics
We've been putting off physics work for a while now. We had kinematics, parenting, a naive broadphase, and sphere-vs-sphere narrowphase, and naive raycasting, but we still needed a lot more work. The broadphase failed spectacularly at large vs small object checking, raycasting needed to be accelerated through the broadphase spatial grid, and we wanted more shapes (convex hulls and boxes) and numerical robustness. We also didn't have a way to actually respond to collisions outside of kinematics (for sparks, sound effects, damage, etc). None of that is particularly scary, but the sheer amount of work still needed was.
Josh decided we should investigate off-the-shelf physics engines and decide which path would strike the right balance of development time and quality. So I took some time to evaluate our options. The 3 most well known being Havok, PhysX and Bullet. Honestly, I don't like the way most Havok demos look and feel. The licensing for PhysX is messy and I'm just kind of assuming it's a behemoth. In all the comparisons I've looked at Bullet seems to be the most consistent. It's rarely the absolute best in terms of performance or simulation quality, but it's very consistently #2 and right on the heels of #1. Physics and Havok on the other hand are excellent in some situations and abysmal in others. The consistency of Bullet is extremely appealing to me.
So I decided to integrate the bare minimum amount functionality leveraging Bullet that would let us see it in action. That would let us gauge the quality in our own use case and the time/difficulty associated with using it. I decided to do this as an entirely separate physics API in the engine that mirrored the existing one but didn't replace it. This let us actually toggle back and forth between new physics and old physics with a single variable. This was fantastic for comparing the two. We were able to ensure that everything felt exactly the same in both engines. This meant we wouldn't have to re-tune ship controls if we did end up swapping out the physics and also highlighted anywhere our current physics implementation was doing things incorrectly. Luckily, nearly everything matched up easily: drag, restitution, friction, etc. However, our inertia was pretty far from accurate. This worried me since it made ships feel completely different. In the end I was able to fix the inertia calculation in the old physics and updating the forces and torques could be done analytically to keep the same feel.
Once everything was set up correctly Bullet ended up feeling identical. Implementing it wasn't particularly difficult, it has all the features we want, and it's reasonably performant. It looked like it would be faster to switch to Bullet and we were confident it would yield very acceptable results. I set out to officially migrate us over and to flesh out the rest of the physics API we need to finish the game.
Unfortunately, nothing is ever easy and I it wasn't long before I started finding Bullet's rough edges, quirks, and flaws. Constraints are unusably slow, compound objects and triggers have awful API, I found a couple outright bugs, memory allocation is a mess, and the documentation is a half step above worthless.
Still, with enough hammering, it works. We have all the features we were aiming for and the implementation isn't a complete disaster. Some of the new stuff we now have is shapecasting, overlap tests, triggers, collision groups and masks, convex hulls and box shapes, and per-system physics. It may have taken 5 weeks, but it's done now and it shouldn't need any major changes going forward.
Overall, it's a win and I think it was the right choice. However, in the future I would likely avoid Bullet. In a professional setting I'd go with PhysX and see if it fares better. On hobby stuff I'd suck it up and write my own. The real kicker though, is that Bullet isn't awful. The author has done some quality work and I'm happy Bullet exists, it just the API design that's a disaster. All it needs is about a month of someone with API design chops hammering on it and it would go from irritating to a joy to use.
Math
Along the way, something happened that brings me great joy. I finally fixed all the math in the engine! This has been sitting in the back of my brain driving my nuts for months now. I probably first realized we had serious issues back in December. It's one of those things the tends to show up at unexpected times when you're in the middle of some other task and fixing it is going to break a ton of other things and you don't even know where it's broken and no one knows how quaternions even work and it's going to take forever and you just want to finish this one task and...you get the idea. It continually gets pushed to the back burner.
When I ran into it again with the camera work in the last update I told Josh that the next time it comes up one of us is going to suck it up and fix it. And by 'one of us' I mean me because Josh hates quaternion and matrix code.
Well, when I was integrating Bullet I ran right back into these math issues almost immediately. When rendering we have to get the position and rotation of objects out of the physics engine. Bullet has a lovely function that fills a matrix in an OpenGL format. But our matrices were in some weird, partially transposed state due to our bugs so we couldn't even use that matrix without first doing some obscene hacks to get it to match our format.
I mentally prepared myself for a good 5 days of anguish digging through our math and tracking down every last issue. I spent a couple hours cataloging all known issues and our coordinate system conventions in every part of the rendering pipeline. In the end, the majority of our issues traced back to Matrix_GetRight/Up/Forward and Quat_GetRight/Up/Forward. Turns out almost all of the math was 'correct' except when we converted from axes to actual on-screen directions. Both sets of functions were wrong and wrong in different ways. This is what made it tricky. Once I realized that I simplified a bunch of our math and found a few other small mistakes and remove all the ugly hacks we had before. The tests I wrote last time were a tremendous help. In the end, I only spent a day on it. Victory.
The End.
And now we get to the sad part. This will be my final dev log. My journey with the awesomeness that is Limit Theory is at an end.
From the very beginning Josh and I discussed a finite end. My job was to help clear the last major hurdles and open up the path for Josh to grind away on gameplay. To that end, I think this has been highly successful. The hope was that we'd be able to ship during that time frame, but alas, we didn't make it.
My plan for the last few years has been to move to the west coast, maybe Seattle, and find work with a stable indie studio. As luck would have it, something decided to fall directly into my lap. Shortly before GDC, Blizzard reached out to me and, long story short, I'm joining their new shared engine team at the beginning of August.
I leaned away from working for a AAA studio because I was scared of the potential soul-sucky nature of it, but once I actually visited Blizzard, that changed. I've gotta say, they've built an amazing culture there. Everyone I spoke with was happier and more creatively empowered than almost every indie studio I've seen. And I can bring Tess to work.
For the past month I've been working part time, winding down. I'm happy I managed to get Bullet fully integrated before I leave. I'm incredibly proud of the work I've done here, and extremely grateful I've gotten to work alongside an awesome programmer and person like Josh. I've grown tremendously as a programmer in my time here, and I certainly wouldn't be in the situation I am if it hadn't been for Procedural Reality. It's been a fantastic 14 months.
It's bittersweet, as endings tend to be. I'll miss the insanely creative and detailed discussions and the uniquely welcoming and cerebral community you've all built. I'm also excited beyond words about what comes next.
Cheers,
Adam
As always, I've been bouncing around like a madman. Sprinkling a little feature dust here, vacuuming a few bugs there, reinforcing scaffolding, ensuring the house stays tidy as we scale, etc etc. Oh, and I may have finished an entire engine system along the way. First up, Docking!
Docking
Last time I showed off the command interface. A large part of that was ensuring we can switch between 'contexts' smoothly. UI gets swapped out, the camera animates, bindings are changed out, and so on. These are small things that are going to be leveraged frequently. After finishing the command interface I wanted to continue working on gameplay and also push on these features a little more to see how they hold up. Docking seemed like a good fit.
We wanted to start with just the core mechanics: fly close to a space station, press a button, auto-pilot to the docking port, and see UI menus for everything you can do at that station.
First, a dockable component. The UI simply searches for dockables and if you're in range presents a button prompt to begin docking. The act of docking actually removes the player's ship from the world and adds it to the station instead. Similarly, once docked an undock prompt is presented.
I extended the MasterControl I implemented last time to add 'control sets'. The UI looks at the player each frame to determine which control set ought to be active and automatically switches to it. Previously we could choose from piloting, commanding, and debug controls. Now, when the player is docked we swap to a control set with things like your storage locker, merchants, and the jobs board. Since we can already swap out UI trees easily this ended up being dead simple to implement.
With all the state control in place, I wanted to make it a smooth, physical docking operation where you literally fly into the station. The AI is implemented through an action stack. Actions are pushed onto the stack and AI simply run their current action each frame, popping them once complete. Conveniently, the player is no different. The action stack is just empty and the UI controls poke the player's state directly. (This actually surprised me when I built the command interface because I could select my own ship along with my allies when giving orders and my own ship would fly along, taking part like any other unit.) I added a new docking action that is essentially the 'move to location' action that also happens to reparent the player from the system to the station once at the destination. And boom, with a whopping 7 lines of code you now auto-pilot right on in.
If a station is destroyed a fraction of the damage is inflicted on docked entities and they're released from the station. If your ship survives the damage, great, you can limp away. If not, well, your attacker gets to rummage through the debris of your ship along with the station.
It's not a particularly big or complicated feature, but I'm thoroughly pleased with how brain-dead easy it was to implement. The way Josh structured AI and actions is excellent. There's a ton of power and flexibility there with no real complexity. The whole of implementing docking took a day, with about 3 hours being the 'real work' and the rest polish and iteration.
Physics
We've been putting off physics work for a while now. We had kinematics, parenting, a naive broadphase, and sphere-vs-sphere narrowphase, and naive raycasting, but we still needed a lot more work. The broadphase failed spectacularly at large vs small object checking, raycasting needed to be accelerated through the broadphase spatial grid, and we wanted more shapes (convex hulls and boxes) and numerical robustness. We also didn't have a way to actually respond to collisions outside of kinematics (for sparks, sound effects, damage, etc). None of that is particularly scary, but the sheer amount of work still needed was.
Josh decided we should investigate off-the-shelf physics engines and decide which path would strike the right balance of development time and quality. So I took some time to evaluate our options. The 3 most well known being Havok, PhysX and Bullet. Honestly, I don't like the way most Havok demos look and feel. The licensing for PhysX is messy and I'm just kind of assuming it's a behemoth. In all the comparisons I've looked at Bullet seems to be the most consistent. It's rarely the absolute best in terms of performance or simulation quality, but it's very consistently #2 and right on the heels of #1. Physics and Havok on the other hand are excellent in some situations and abysmal in others. The consistency of Bullet is extremely appealing to me.
So I decided to integrate the bare minimum amount functionality leveraging Bullet that would let us see it in action. That would let us gauge the quality in our own use case and the time/difficulty associated with using it. I decided to do this as an entirely separate physics API in the engine that mirrored the existing one but didn't replace it. This let us actually toggle back and forth between new physics and old physics with a single variable. This was fantastic for comparing the two. We were able to ensure that everything felt exactly the same in both engines. This meant we wouldn't have to re-tune ship controls if we did end up swapping out the physics and also highlighted anywhere our current physics implementation was doing things incorrectly. Luckily, nearly everything matched up easily: drag, restitution, friction, etc. However, our inertia was pretty far from accurate. This worried me since it made ships feel completely different. In the end I was able to fix the inertia calculation in the old physics and updating the forces and torques could be done analytically to keep the same feel.
Once everything was set up correctly Bullet ended up feeling identical. Implementing it wasn't particularly difficult, it has all the features we want, and it's reasonably performant. It looked like it would be faster to switch to Bullet and we were confident it would yield very acceptable results. I set out to officially migrate us over and to flesh out the rest of the physics API we need to finish the game.
Unfortunately, nothing is ever easy and I it wasn't long before I started finding Bullet's rough edges, quirks, and flaws. Constraints are unusably slow, compound objects and triggers have awful API, I found a couple outright bugs, memory allocation is a mess, and the documentation is a half step above worthless.
Still, with enough hammering, it works. We have all the features we were aiming for and the implementation isn't a complete disaster. Some of the new stuff we now have is shapecasting, overlap tests, triggers, collision groups and masks, convex hulls and box shapes, and per-system physics. It may have taken 5 weeks, but it's done now and it shouldn't need any major changes going forward.
Overall, it's a win and I think it was the right choice. However, in the future I would likely avoid Bullet. In a professional setting I'd go with PhysX and see if it fares better. On hobby stuff I'd suck it up and write my own. The real kicker though, is that Bullet isn't awful. The author has done some quality work and I'm happy Bullet exists, it just the API design that's a disaster. All it needs is about a month of someone with API design chops hammering on it and it would go from irritating to a joy to use.
Math
Along the way, something happened that brings me great joy. I finally fixed all the math in the engine! This has been sitting in the back of my brain driving my nuts for months now. I probably first realized we had serious issues back in December. It's one of those things the tends to show up at unexpected times when you're in the middle of some other task and fixing it is going to break a ton of other things and you don't even know where it's broken and no one knows how quaternions even work and it's going to take forever and you just want to finish this one task and...you get the idea. It continually gets pushed to the back burner.
When I ran into it again with the camera work in the last update I told Josh that the next time it comes up one of us is going to suck it up and fix it. And by 'one of us' I mean me because Josh hates quaternion and matrix code.
Well, when I was integrating Bullet I ran right back into these math issues almost immediately. When rendering we have to get the position and rotation of objects out of the physics engine. Bullet has a lovely function that fills a matrix in an OpenGL format. But our matrices were in some weird, partially transposed state due to our bugs so we couldn't even use that matrix without first doing some obscene hacks to get it to match our format.
I mentally prepared myself for a good 5 days of anguish digging through our math and tracking down every last issue. I spent a couple hours cataloging all known issues and our coordinate system conventions in every part of the rendering pipeline. In the end, the majority of our issues traced back to Matrix_GetRight/Up/Forward and Quat_GetRight/Up/Forward. Turns out almost all of the math was 'correct' except when we converted from axes to actual on-screen directions. Both sets of functions were wrong and wrong in different ways. This is what made it tricky. Once I realized that I simplified a bunch of our math and found a few other small mistakes and remove all the ugly hacks we had before. The tests I wrote last time were a tremendous help. In the end, I only spent a day on it. Victory.
The End.
And now we get to the sad part. This will be my final dev log. My journey with the awesomeness that is Limit Theory is at an end.
From the very beginning Josh and I discussed a finite end. My job was to help clear the last major hurdles and open up the path for Josh to grind away on gameplay. To that end, I think this has been highly successful. The hope was that we'd be able to ship during that time frame, but alas, we didn't make it.
My plan for the last few years has been to move to the west coast, maybe Seattle, and find work with a stable indie studio. As luck would have it, something decided to fall directly into my lap. Shortly before GDC, Blizzard reached out to me and, long story short, I'm joining their new shared engine team at the beginning of August.
I leaned away from working for a AAA studio because I was scared of the potential soul-sucky nature of it, but once I actually visited Blizzard, that changed. I've gotta say, they've built an amazing culture there. Everyone I spoke with was happier and more creatively empowered than almost every indie studio I've seen. And I can bring Tess to work.
For the past month I've been working part time, winding down. I'm happy I managed to get Bullet fully integrated before I leave. I'm incredibly proud of the work I've done here, and extremely grateful I've gotten to work alongside an awesome programmer and person like Josh. I've grown tremendously as a programmer in my time here, and I certainly wouldn't be in the situation I am if it hadn't been for Procedural Reality. It's been a fantastic 14 months.
It's bittersweet, as endings tend to be. I'll miss the insanely creative and detailed discussions and the uniquely welcoming and cerebral community you've all built. I'm also excited beyond words about what comes next.
Cheers,
Adam