Return to “Dev Logs”

Post

[Josh] Friday, August 17, 2018

#1
Friday, August 17, 2018

Hey everyone!

It's been a good week and, thankfully, I finally caught a break in my work on moving UI engine-side. Today's log will get a bit technical, but this is an interesting topic and I'd like to detail my experience with it in the hopes that it may be of use to other developers who face the same decision in their work.

Strap in, it's a long one...

GUI Architecture: Immediate or Retained?

I mentioned in last week's pseudo-log that there are effectively two major approaches to architecting a UI. The two (very different) styles are known as immediate mode (IM) and retained mode (RM). RM is more traditional and probably what you're used to if you've used a GUI framework directly from code before. IM, however, has enjoyed a reasonable amount of attention in the past few years, and I have seen more and more developers asking about the viability of it. This is no doubt in part due to Omar Cornut's excellent Dear ImGui library, which has demonstrated to many (including myself) the tremendous power that the IM paradigm offers for creating interfaces with minimal fuss.

Briefly, one can illustrate the difference in approach by glancing at some example usage code. Here's what some typical RM pseudocode would look like:

Code: Select all

g = GUI.Canvas()
g.add(GUI.Window("Universe Configuration")
  .add(GUI.Checkbox("Infinite Mode").bindTo(&config.infinite))
  .add(GUI.Slider("Average Systems Per Region", 10, 1000).bindTo(&config.regionSize))
  .add(GUI.Slider("Average System Connectivity", 1, 10).bindTo(&config.connectivity))
  .add(GUI.GroupHorizontal()
    .add(GUI.Button("Create", createUniverse))
    .add(GUI.Button("Cancel", cancelUniverse))
  )
)

...

g.update()
g.draw()

Again, that's just pseudocode, and there are a million ways to structure such APIs for convenience, but the general idea is that interface pieces are created like objects, and we control the UI by calling various functions on those objects. RM style is very much a classic, object-oriented approach to UI.

On the other hand, have a look at what IM looks like:

Code: Select all

GUI.BeginWindow("Universe Configuration")
  GUI.Checkbox("Infinite Mode", config.infinite)
  GUI.Slider("Average Systems Per Region", 10, 1000, config.regionSize)
  GUI.Slider("Average System Connectivity", 1, 10, config.connectivity)
  GUI.BeginGroupHorizontal()
    if (GUI.Button("Create")) createUniverse()
    if (GUI.Button("Cancel")) cancelUniverse()
  GUI.EndGroup()
GUI.EndWindow()

At first glance this may look similar, but, in fact, the IM approach could hardly be any more different! Here we are not dealing with objects. Instead, we're just making function calls. An immediate mode interface is inherently interwoven with the interface's data and logic. No widget demonstrates this more aptly than a simple button, which is perhaps the most famous example of the difference between IM and RM. Whereas an RMGUI button typically involves setting a 'callback' function (which the interface will call if the button is pressed), an IMGUI button is actually a function that returns true if pressed.

To really appreciate the massive ramifications of the choice between RM and IM requires that you implement a significantly complex interface in both styles. Luckily, I have done so, and will therefore lay out the most prominent pros/cons that I have found in my own interface-building work. Unfortunately, it turns out to be the case that both approaches come with very real pros and cons, and neither, in isolation, is ideal.

Immediate Mode

Both the strengths and weaknesses of IMGUIs come directly from the fact that a vanilla IMGUI does not exist separately from the data that it controls and the logic it executes.

Pros
  • UI code is extremely concise, easy-to-understand, and quick to write
  • The UI does not need any 'extra information' -- sliders/checkboxes/radio groups don't need pointers to the data they represent, buttons don't need callbacks for the actions they trigger
  • Changing data never requires special logic to inform the UI about changes, making it effortless to build UI for dynamic data that changes frequently (like game data!)
  • Easy to use from scripts; no special binding code necessary

Cons
  • Automatic layout is heavily-restricted
  • 1-frame delays are common and sometimes unavoidable; can result in 'popping' and added input latency
  • Higher CPU usage

Now, please note that each of these cons is very sensitive to implementation details and requires a lot more explaining to understand the full story, so please don't take the above list as a good "generalization" of IMGUIs. Especially the last point -- I have seen a great deal of misinformation about immediate mode performance characteristics online. Much of the difficulty in IMGUI implementation is concerned with mitigating these cons in various ways. That being said, it is fair to say that complex layout is a rather fundamental problem for a traditional IMGUI implementation.

Retained Mode

Again, the strengths and weaknesses of RMGUIs are a direct consequence of the fact that an RMGUI is built from first-class objects that exist in and of their own right.

Pros
  • Automatic layouts are easy and can be made arbitrarily-complex
  • No frame delays, minimal input latency
  • Minimal CPU usage (predicated, of course, on a well-optimized implementation)

Cons
  • UI code is often verbose
  • UI must have knowledge about the data it is displaying and must understand how to trigger functionality -- pointers and callback mechanisms are typical
  • Users must take special care to ensure the UI stays in-sync with data -- complex, frequently-changing data requires significant consideration to work properly
  • Usage from scripts requires special consideration; UI must know how to access script data & functions

Stuck Between an Asteroid and a Hard Place

Looking at the pros/cons of each paradigm, it's not hard to see that IMGUIs and RMGUIs are effectively polar opposites. Where one excels, the other lacks, and vice-versa. It is the absolute epitome of a difficult and highly-consequential trade-off.

Last week, I implemented an IMGUI in our engine. It was my first time implementing an IMGUI, and, while I understood the cons beforehand, I didn't know how they'd pan out in practice -- how well could I mitigate them? The answer turned out to be: quite well, but still not well enough. In particular, the restrictions on automatic layout forced a no-win choice between having to litter my UI code with fixed sizes, or having to accept noticeable, 1-frame-long pops / delays. It is an absolutely fundamental limitation of a standard IMGUI. I believe I used the word "heartbreaking" in one of my posts last week, and it was not an exagerration. The IM paradigm affords such incredible easy and clarity in creating game UI, yet the drawbacks were too much to stomach.

On the other hand, I've implemented RMGUIs many, many times before. I'm more than familiar with those cons. The added complexity in using RMGUI from script is, for me, something to avoid at all costs. After all, the impetus for this recent effort to move our UI code to C stemmed from a burning desire to view & interact with gameplay mechanics with minimal pain. For me, the RM paradigm flows counter to that goal!

If only we could have all the things. If only we could immedify our retained mode, or retain our immediate mode. If only.

As you've already guessed, it turns out: we can :)

Hybrid Mode GUI

Thus far, my discussion of IMGUIs has been rather specific to what I've called a 'standard' or 'vanilla' implementation. That's because, in reality, one can go much, much further under the hood of an IMGUI in order to mitigate or even defeat the stated weaknesses. At some point, the line between immediate and retained can start to blur...and in that lovely gray area lies the answer to all our problems. The paradigm that I will describe to you now could be considered as simply an 'advanced' IMGUI implementation, however, for the sake of clarity, I will call it 'hybrid mode' -- HMGUI :nerd:

Much of the beauty of IMGUI comes from the friendliness of the user-facing API. Calling a sequence of functions to implicitly map an interface onto a set of data and functionality is simply easier than creating explicit constructs to do so. At the same time, trying to perform standard internal GUI work like layout and input handling without having advance knowledge of the entire interface results in the inherent restrictions discussed above. But suppose we were to 'retain' all necessary information from our 'immediate'-style functions, and defer that internal work to after the entire interface has been specified? It's a winning combination.

The basic premise of an HMGUI is that, each frame, we will build a somewhat-traditional widget hierarchy under the hood as the user is issuing IM-style calls. We'll retain just enough information to be able to perform automatic layout on the UI and to handle input later. Once the user has finished calling into the API, we'll go back through our hierarchy and perform all the standard GUI logic: layout, input handling, and whatever else.

As a consequence of deferring the work, hybrid mode can handle the full gamut of complex, automatic layout functionality that one would expect from retained mode. We can have widgets stretch to fill available area, align themselves within a group, automatically compute group sizes, etc -- all without specifying explicit sizes and without a one-frame delay (one of which would be necessary under a pure IMGUI). We can have selectable widgets like buttons respond to mouse-over immediately, minimizing perceptual latency. We can layer widgets in arbitrarily-complex ways, performing on-the-fly z-reodering without fuss.

So...where's the catch? Surely there can be no free lunch. Well, HMGUI isn't really a free lunch: of the retained, immediate, and hybrid paradigms, hybrid mode takes the most work to implement. That's not surprising when you consider that, under the hood, HM is just a clever mixing of IM and RM, thus requiring much of the implementation work of both. That being said, from the perspective of the user of a hybrid API, it really is a free lunch :) And if you know me, you know that's exactly the kind of system I love. Push all of the hard work into the engine/systems, leave the game/application code as clean and simple as possible.

One might also point out that, of the three, HM consumes the most CPU time, due to the fact that it involves all of the CPU work of IM plus the layout work of RM. However, in practice, such code can be made so blazingly-fast that the point is moot (especially when written in well-optimized C ;) ). As always, performance or lackthereof is almost entirely the result of the implementation quality.

A Few Examples

I haven't implemented the more complex widgets yet, as I focused heavily on core details this week. I also haven't worked much on graphics. As we all know, making things shiny is a beloved hobby of mine, but best saved for...later :oops: Still, even with only basic widgets and fairly rudimentary rendering, a close look at HMGUI already reveals the superiority.

Take, for example, my little todo list from last week's IMGUI:

Image

There are a number of annoyances here. Checkboxes aren't correctly aligned with text. That's my fault, not a limitation of IMGUI, but it happened because writing the IMGUI code to manually align and lay things out is quite a tedious and error-prone endeavor. The code that shows this list is littered with 'magic' size constants -- the window width, for example, is a constant and would not grow based on the contents. Again, in IMGUI we have to accept such constants or the frame delay problem. Despite the simplicity of an immediate mode API, trying to achieve a polished, consistent look can quickly turn the code messy.

From this week's HMGUI:

Image

Consistent, polished, and the code for creating it is cleaner thanks to the fact that all the layout work is handled automatically. Everything is aligned, padded, and spaced with precision. I have even swapped the checkbox to right-justified to demonstrate automatic stretching (again, problematic for IMGUI). This window is sized automatically and will grow accordingly should I add new, longer todo items.

A bigger example:

Image

Sorry again for the rough graphics...but the beauty here is in the functionality. This stream-of-consciousness-style test window has more automatic layout going on than you can shake a stick at! This one is really not going to happen in a vanilla IMGUI implementation. At least, I wouldn't want to see the code for it :shock: In HMGUI, however, it's absolutely straightforward. Notice how even the embedded todo list has expanded the checkbox elements slightly due to the fact that the split code view on the bottom is dictating the window's width. It's all in the details! :)

Conclusion

GUI paradigms present a difficult choice for developers. The simplicity of an immediate mode API is tantalizing. Creating interfaces is a breeze, and the resulting increase in productivity should not be taken lightly. On the other hand, retained mode offers precise control over complex layouts that are difficult if not impossible to achieve in immediate mode. By combining the front-end elegance of the IM paradigm with the back-end power of RM, we can, thankfully, have the best of both worlds! Hybrid mode GUI is the way to go :)

I'm very satisfied to have finally found some success with this work, and I'm glad that I took the time to experiment with a new paradigm, as I would never have come to this solution without knowledge of both. Always a treat when failure leads to reward. Although the feature set of this new HMGUI implementation is still rather slim and the aesthetics quite programmer-artsy, the foundation is layed and the road has been paved. I already have enough power to get back to doing what I wanted to do in the first place: move forward with gameplay interaction. I'll be continuing the implementation of more advanced features as the need arises, so I'm sure we'll be hearing more about HMGUI in the future. For now, I'm happy to call the porting of another major system to C a success, and excited to move back into gameplay work with my new toys :D

Enjoy your Friday! :wave:
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: [Josh] Friday, August 17, 2018

#2
Sorry again for the rough graphics...
Um what? if that's rough graphics I can't wait to see polished, but as discussed in the previous log do that in Beta :) This looks perfectly fine, and I'm glad your experiments have also made it functional and practical and end-user oriented. Modding will be a joy! :geek: :D

A couple things I noticed:
Image
is this something behind the scenes or something for communication between players and NPCs?

Image
!!!!!!! LT will be playable on a gamepad?
For now, I'm happy to call the porting of another major system to C a success, and excited to move back into gameplay work with my new toys :D

Enjoy your Friday! :wave:
You have yourself a good 4 day weekend to relax and get away from the code for a bit, go fishing, play laser tag, go to the beach, hang out in your underwear and binge a season of something sci-fi. That's an order![/Dr. Hyperion] :thumbup:
Image
Challenging your assumptions is good for your health, good for your business, and good for your future. Stay skeptical but never undervalue the importance of a new and unfamiliar perspective.
Imagination Fertilizer
Beauty may not save the world, but it's the only thing that can
Post

Re: [Josh] Friday, August 17, 2018

#6
I doubt if you will be surprised to read that after skimming through the bulk of your dev log, Josh, I tended to concentrate on your "Conclusion". I understand that what you've done with the GUI will make a worthwhile contribution to the game I will eventually be playing, and I thank you for that. :)
JoshParnell wrote:
Fri Aug 17, 2018 6:49 pm
For now, I'm happy to call the porting of another major system to C a success, and excited to move back into gameplay work with my new toys :D
This one line was the cause of much joy for me. To read that you are excited about returning to the gameplay work answers a previous question which was likely not seen by you. I hope you have an enjoyable weekend. :D :angel:
Post

Re: [Josh] Friday, August 17, 2018

#7
This may seem like a stupid question to those of you au fait with games programming, but as a web developer I have to ask why HTML/CSS is not the solution here. It's a very mature design with more than a decade of competent people working on it in order to do exactly this, create aesthetically pleasing and functional UIs. Instead of JavaScript, you could just use lua. I understand that now you probably won't go back and change this, but I'd like to know why this wasn't an option when you began your search for a solution.

I am very glad that you're getting back to gameplay again, and that this problem is done and partially dusted.
A life well lived only happens once.
Seems deep until you think about it.
Post

Re: [Josh] Friday, August 17, 2018

#8
vector67 wrote:
Sat Aug 18, 2018 9:34 am
This may seem like a stupid question to those of you au fait with games programming, but as a web developer I have to ask why HTML/CSS is not the solution here. It's a very mature design with more than a decade of competent people working on it in order to do exactly this, create aesthetically pleasing and functional UIs. Instead of JavaScript, you could just use lua. I understand that now you probably won't go back and change this, but I'd like to know why this wasn't an option when you began your search for a solution.
While that solution is certainly not the most performant, it indeed has been used effectively in many AAA games. One explicit behind-the-scenes about that is this: http://twvideo01.ubm-us.net/o1/vault/gd ... lement.pdf
Post

Re: [Josh] Friday, August 17, 2018

#11
Flatfingers wrote:
Sun Aug 19, 2018 2:49 am
A thing of beauty.

Also, is there any part of Limit Theory that, once the game is done, won't be worth a white paper or technical article? :D So many interesting problems solved!
I imagine that after release, the devlogs will be a rich vein of information and inspiration for other developers. I can't think of many who go into so much depth so regularly, giving a very detailed account of how the sausage was made.

Also, I just noticed the most terrifying statement an NPC would ever encounter
(GUI.Button("Cancel")) cancelUniverse()
Image
Challenging your assumptions is good for your health, good for your business, and good for your future. Stay skeptical but never undervalue the importance of a new and unfamiliar perspective.
Imagination Fertilizer
Beauty may not save the world, but it's the only thing that can
Post

Re: [Josh] Friday, August 17, 2018

#12
Hyperion wrote:
Sun Aug 19, 2018 5:20 pm
Flatfingers wrote:
Sun Aug 19, 2018 2:49 am
A thing of beauty.

Also, is there any part of Limit Theory that, once the game is done, won't be worth a white paper or technical article? :D So many interesting problems solved!
I imagine that after release, the devlogs will be a rich vein of information and inspiration for other developers. I can't think of many who go into so much depth so regularly, giving a very detailed account of how the sausage was made.
There was one, Hyperion.
John Carmack, back when his "finger" logs (Unix rulez!), would go into great detail about Doom and Quake.
Why they had to use David Abrash's assembly skillz to make the engine performant (back before the day's off dedicated 3d hardware that we take for granted today).
:D

:clap:
A joyous day, Josh, one that I'm very happy you took the time out to review and rewrite the books on.
:clap:
YAY PYTHON \o/

In Josh We Trust
-=326.3827=-
Post

Re: [Josh] Friday, August 17, 2018

#13
Hyperion wrote:
Fri Aug 17, 2018 7:22 pm
Image
!!!!!!! LT will be playable on a gamepad?
JoshParnell wrote: probably because I've actually held a controller in my hand and played the thing
since 2014

Also, why not? Just a generic input device, if you have any non kb+m input devices supported gamepads are easy.
Last edited by Cornflakes_91 on Mon Aug 20, 2018 3:20 pm, edited 1 time in total.
Post

Re: [Josh] Friday, August 17, 2018

#14
Cornflakes_91 wrote:
Mon Aug 20, 2018 4:15 am
Hyperion wrote:
Fri Aug 17, 2018 7:22 pm
Image
!!!!!!! LT will be playable on a gamepad?
JoshParnell wrote: probably because I've actually held a controller in my hand and played the thing
[Url=
viewtopic.php?f=12&t=2829&p=46781&hilit ... ler#p46781
since 2014[/quote]

Also, why not? Just a generic input device, if you have any non kb+m input devices supported gamepads are easy.
[/quote]

Yeah, could even play it with the Vive Wands, though it'd be awkward to get the mapping right, and you wouldn't be able to use most of the features without VR code also in there.
°˖◝(ಠ‸ಠ)◜˖°
Toba - A Development Dump
Post

Re: [Josh] Friday, August 17, 2018

#15
Great post as usual, Josh!

I just wanted to share my opinion on the whole RM vs IM debacle since I have recently implemented a somewhat complex and fully responsive UI using dear imgui internal framework (only using dear imgui behaviors and fully custom styling and layouting). Basically, not to take away from your achievement, but you've implemented what dear imgui already had from the beginning. Dear imgui retains a fair bunch of info about windows so you can query their sizes before laying out your elements. Regarding to a frame delay: I'm sure there are many cases where this is a problem, but in a common case where, for example, you have a button drawn last which is supposed to toggle visibility of an element drawn first what you could do is discard the whole UI on click and recompose it again all within the same frame. My UI takes about 0.5ms of CPU time per frame so it's not a huge deal.

Online Now

Users browsing this forum: No registered users and 1 guest

cron