The State of Limit Theory Development, 2017 Edition!

Read and discuss the latest big updates in the development process.

The State of Limit Theory Development, 2017 Edition!

Postby JoshParnell » Sun Jan 22, 2017 10:29 pm

Hello and good evening lovely Limit Theorists!

Warning: please ensure that your seat-belts are fastened, cockpits fully-pressurized, and pulse weapons overloaded to at least 110% of standard operating power. It's going to be a long ride through this sector.

~ Introduction ~

    First off, I've got to start by saying that this post is coming to you several months too late, but, as I'll discuss later, I'm taking serious steps to ensure that I change my communication habits. I'm sincerely sorry for not being on the forums for several months; I know, coming back and reading some of the intermittent posting, that it's taken a toll on many of you guys' outlook on the game. With this post, I aim to lift that outlook, but also to open up the hood and talk about why things are the way they are at the moment.

    Before we get down to it, a word about my previous updates and the community's response. If you're one of the many posters who has said something to this effect, know that I've heard you when you said "no more vagueness," "concrete details," "more than promises/empty words," etc. I got it, my previous updates were not satisfactory, but give me a chance to prove to you that I can, in fact, change :) On the other hand, a forewarning: to those who have demanded "pictures/videos or LT is kill," let me tell you up-front that you're going to be disappointed. I'll explain in detail why that's not what development is about right now, and this is the part where you all are going to have to meet me halfway in understanding that progress and pretty screenshots are NOT equivalent -- a mistake to which I've fallen prey for much of LT's development.

    The fact that development is not fun, pretty, or exciting (at least to most) is part of the very reason for which I haven't been excited to talk about it, and have even avoided doing so. For years I had beautiful things to throw up in the air and say "LOOK WOT I DID!" and receive praise for making things shiny. Man I miss those days so, so much...but the problems I'm dealing with currently are far more pressing to LT being released than more visual candy. I simply have to push through the problems I'm about to explain if you want LT to be finished. And I have every intention of LT being finished. There will be more pretties, more videos, etc., but let me be very clear: that won't happen before I get past the current roadblock.

~ Development: Where We've Been, Where We Are, Where We're Going ~

    To understand why development has taken the bizarre course that it has, I must take you back all the way to just before the Dark Days, when the 'Road to the Beta' was in full-swing and Josh was burning his candle at 17 different ends. Things seemed to be moving quickly and beta seemed within reach. Then, all of the sudden, the Dark Days were upon us. What exactly happened, other than stress and an unhealthy lifestyle pushing me over the brink? What happened is that I was backed into a corner from which there was no easy escape. I had been backing into this corner for quite some time without realizing it.

    LT was, at that time, a massive codebase. Not massive in relation to how much it did, but still massive with respect to your average programming project. With both C++ and GLSL included, the code was pushing dangerously-close to 100,000 lines. Interestingly, 100K lines is a bit of a special number in programming. It's one of those rule-of-thumb things, but a large number of programmers, now including me, have found it to be roughly accurate: they say that a skilled programmer can manage about a hundred thousand lines of code. What do I mean by that? Think of it like RAM: your computer has a certain amount of RAM. When a program tries to use more memory than you have RAM, typically the program will still function via an OS mechanism called 'paging,' wherein hard drive space is used to effectively extend the limits of RAM. Problem is, doing so comes at a huge cost (less so with SSDs, but still a high cost): RAM accesses now may take orders of magnitude longer due to needing to page in/out memory from/to disk. Sure, your program will still run, but the moment you hit your RAM limit, a typical program will almost immediately incur a huge performance penalty. And so it is with programmers! Good programmers have a 'RAM' of about 100K lines of code. Beyond that, the difficulty of understanding and maintaining said code quickly becomes disproportionately hard, leading to a harsh dropoff in productivity.

    So, I was starting to approach the limits of my programmer RAM. My solution to this problem (as well as modding) was LTSL. LTSL made things gloriously easy to understand, and, for a while, made development gloriously fast in comparison to how C++ development had been. There was a brief period of time wherein I found myself basking in the sun and all was right with the LTverse. LTSL was making development a joy, I was able to keep everything in my mental RAM thanks to the compactness of the language...things were great. Until, one day, they weren't.

    You see, while I had successfully evaded one wall, another was sneaking up on me, and I would soon be caught between a rock and a hard place. LTSL, like all non-compiled scripting languages, was slow. Not terribly slow, but certainly slow in comparison to C++. This was the price I had to pay for offloading gameplay code to LTSL. I didn't anticipate that the price would be too much. On that point, I was quite wrong. It was a rather sudden thing, due to the nature of Limit Theory -- there's so much going on at all many game entities, so many UI components, so much AI, so much physics, so much rendering -- speed degradation of only a small bit of code can cause a frightening FPS hit, since that code may be running every frame on every entity that, just for example, emits an EM signature that can be picked up via scanner. As I pushed into my beautiful new world of C++ & LTSL, I was also pushing closer to the invisible wall of performance problems, and then, one day, I hit it: FPS was down to ~20 even in situations where a relatively small amount was going on (i.e., even without LOD system simulations active, without distant AI, etc.) I knew I was in trouble, because I knew it was just the tip of the iceberg, and that performance was going to quickly go to hell in a handbasket if I continued using LTSL, my saving grace. On the other hand, my inability to mentally manage and make progress on a near-100K LOC codebase in C++ awaited me if I were to turn back. In either direction, I faced a sharp drop-off, either of productivity or performance. I was in a truly impossible situation, and for once, my Crazy-Solution-Generator-3000 failed me. Cornered and out of options, with a community expecting an imminent RTB post, my kernel crashed. And it crashed hard.

    Therein lies the rub, the truly monumental challenge at the heart of building LT (a challenge so formidable that I've given it the title of The Fundamental Problem of Limit Theory; FPLT for short): the completion of Limit Theory demands the utmost efficiency in both run-time performance (a la C++), AND development cost (a la LTSL). These two interests, as most programmers know, are in oppisition to one another in the world of development. In general you can have one, not both. But, you see, with an enormous amount of gameplay logic to crunch on an enormous number of entities, and only one programmer with one mental RAM bank on which to make that all happen, Limit Theory requires both. Voila, FPLT.

~ Solving the Fundamental Problem of Limit Theory ~

    While I've worked on a lot of areas of LT over the past two years (since the Dark Days), >95% of my effort has been devoted to solving FPLT. This has made for a very, very frustrating two years as I've bashed my brain against the hardest problem of my programming career. I've explored numerous solutions. Each prospective solution requires a tremendous amount of effort to even get to a point where I have a simulation large enough to know whether or not said solution will be viable (i.e., I have to build a large-scale, component-based entity simulation with AI to know whether a solution is both compact and performant enough to be an answer to FPLT). So far, the answer has been 'no' to each. Despite all of my accumulated knowledge of code, performance, etc., it's still far too difficult for me to simply intuit whether the prospective solutions will be viable, hence why testing is the only option. It's been a slow, painful process of trial-and-error.

    I promised details, so let's talk about some of my concrete work over the past two years. I've seen posts implying that I haven't been working, and honestly, I can't let that rumor stand. I've been working entirely too hard to get no credit for it, so I'm finally ready and willing to pony up the gruesome details, even though it goes against my long tradition of feeding you all nothing but unicorns and cupcakes :) So here it is...the good, the bad, and of course, the ugly. Put your gloves on.

    First, naturally, there was an attempt to rescue the existing LT by amping up LTSL with a performance overhaul. A practical move at the time. It was a relatively fast failure. I optimized, I changed the way LTSL runs, I molded the memory access patterns from 'my RAM is crying' to 'it could run on punch cards!' Still, I could see that the numbers weren't going up. The gulch between the statically-compiled and intensely-optimized C++ and the runtime-compiled LTSL evaluation engine was too wide to make a dent. Despite my efforts, I'm a bit embarassed to admit that I wasn't ever able to improve LTSL's performance dramatically over baseline, even after blasting it with some of my best optimization cannons.

    With the most practical option shot down, I switched attack vectors and tried what seemed to me the next most practical: splitting the C++ into smaller bits to make it more manageable. I split up what was a 'monolithic' engine, compiled all into one library, into component pieces compiled into different libraries (libltcore, libltrender, libltui, libltai, etc.); this was an attempt to increase both runtime efficiency and my ability to keep all code in RAM. This attempt, luckily, only cost me a few months, as I quickly learned that code is code, and the programmer RAM limit applies no matter how you divvy up the lines. Another solution shot down, another lesson learned.

    I then moved on to a more dramatic change, recognizing that, clearly, the C++ core was simply too big. I was going to have to reduce. Thus ensued one of the most dramatic shifts in LT's development history, as I carefully plucked pieces from the existing codebase and brought them over to a new, 'minimal' C core library. The general idea was to do the really performance-critical stuff in C, do the rest in a still-fast but high-level language (side note: I say C because, by this point in my programming life, I had basically abandoned most C++ constructs, so moving completely to C was not a big issue -- C libraries are friendlier when it comes to linking with the outside world, which is what I was planning to do with respect to the high-level language). My language of choice? Python. It took quite some time, but I eventually engineered the salvaged 'minimal' core of LT, which was a very reasonable 20K lines, along with a cluster of (concise and easy-to-write) Python scripts that could leverage the LT core library for all of the performance-critical and gpu-related stuff. It was quite beautiful. Of course, I knew what I was getting in to: Python is slow. On the surface, it might not look too different from the old LTSL solution. But, unlike LTSL, Python has a massive ecosystem surrounding it. For performance, there's pypy, Cython, Psyco, Pyrex, etc. Much to my chagrin, none of them suited my needs (pypy looked promising, but simply didn't give a big enough speed boost). So I pursued a custom JITing solution. I wrote an intermediate-representation, an AST translator that took Python code to my IR, and an emitter that took my IR to C, which could them be JIT compiled in-memory and executed as machine code by the wonderful TCC. It was a beastly feat of code. In the end, it broke my heart when this solution also failed, not due to performance reasons, but because, as it turns out, compiling Python to a static language (or an IR intended for emission to static language) is indeed near-impossible. Too difficult for me, at least. My JIT worked for simple and moderately-complex logic, but it simply couldn't handle the 'meaty' gameplay logic of LT, as the problem of type inference and other fun things started to become intractable.

    On the bright side, not all was lost with the valiant Python attempt. I did lose quite some time (perhaps on the order of six months) to it. From it, however, I had gained the minimal core onto which I have continued to build (with minimalism!) to this day. The C core will be part of the solution to FPLT. I now have that piece of the puzzle. For the other piece, the 'driver' that will control the core, I'm still searching.

    That brings us up to the present. At this time, my current attack vector is twofold.

    I've been building a solution involving LuaJIT for several months now and am nearing the point where I can do a feasibility test. While I've learned to temper my expectations and wouldn't say that I'm confident that LuaJIT will be the answer, I can say that it's by far the fastest scripting solution I've found. My initial tests, indeed, the only reason I began pursuing a solution in Lua (which wasn't really on my radar), demonstrated that LJ achieves incredible speeds on limited-scope performance tests, even nearing the speed of my own, hand-written C. That's not terribly surprising, since LJ is a genuine JIT, meaning that it's effectively turning Lua into assembly/machine code. It seems to be darn good at doing so. That being said, we'll see how it goes with a full-blown test. I have my reservations about Lua being garbage collected, and I fear that memory access, if anything, may make LJ infeasible. Nonetheless, LJ itself is a formidable feat of engineering, and even if it doesn't end up solving the second piece of the FPLT puzzle, I have a feeling it may still contribute somehow to the final answer (this has happened with so many other 'failed' ideas throughout the course of LT that these days, I almost expect a failed attempt to end up being useful in some way in the future).

    On the other front, I've been building a solution that involves generating C++ via a meta-language based on Python. This would provide a means of performing a lot more of the game logic in ultra-fast compiled code without me having to write 100K lines. In order to get LT working in as few lines as I did previously, I had to use a HUGE amount of complex C++ trickery. While it meant far fewer lines to keep up with, it also made the code more complicated and, in a few cases, sub-optimal in terms of performance. With a meta-language, we can easily solve these problems: I don't care how much code is generated, I only care how much I have to write to generate it. So, for example, whereas setting up the component-based entities in the old C++ engine took several layers of advanced C++, making the component logic hard to understand (boy was this a nightmare with the 'market' component...), I can generate equivalent code in far fewer lines of simple Python. As a bonus, the generated code compiles and runs faster because it's not having to use trickery to reduce code size...I can get Python to spit out tens of thousands of lines in milliseconds, based on ~100 lines of my meta-language. This solution is only a few weeks old, and is sort of a 'nuclear' option in the event that the LuaJIT solution doesn't work out. I'm fairly sure that I can get this approach to work, because it's kind of brute-force. I'd rather be able to find a more elegant way, but if this is what it comes down to, so be it...above all else, I just want to get Limit Theory released at this point, and I'm willing to use whatever I have to use to get there. (By the way, this is totally differenet from the other solution involving Python. Whereas the idea there was to convert Python to machine code on-the-fly, the idea here is simply to use Python as an easier way to write / automate the writing of C++. Kind of like...writing the code...procedurally!)

    So there you have it...the past two years of my life. Has it been shiny? No. Has it been as exciting as the first two years of LT dev? No. But the hard truth is that Limit Theory is an exceptionally-ambitious project, especially considering that we can't just brute force the development with a massive team and all the standard big-budget amenities. Yes, I was young and optimistic when I called it 'easy.' My mistake :? LT is a hard problem. At the center of it all lies FPLT, which encapsulates the problem of needing to build something complex with minimal resources. It's a solveable problem; of this I'm certain. The solution? No, of this I'm not yet certain, but, as I hope to have demonstrated by laying all of my attempts out on the table in detail, I really am attacking it with everything I've got. I can't sugarcoat this situation as I've tried to do in the past, because it just leads to disappointment, so let me be perfectly honest: I don't know when I'll overcome FPLT. I sure as hell hope that it's soon, because two years is already an exceptionally-long time to be beating one's head against the same problem. What I do know is that every failure gets me closer. I learn more about what works and (perhaps mostly..) what doesn't with respect to performance in large-scale simulations. I learn more about low-level performance both of static and dynamic langauges, which helps me to evaluate future solutions. I become adept with more languages, more technologies, hence have an ever-expanding arsenal with which to blast away at development problems (FPLT can only take so much DPS before it breaks. It's a tough tank, but every month my DPS output goes up. It's only a matter of time...) In the mean time, I continue to develop the LT core C library as much as possible to ensure that when I do come to the right solution, all of the underlying machinery is in place for me to quickly and easily plop the existing LT functionality into the mold of said solution.

    Is anyone still listening? :shifty: :ghost:

~ Re-Establishing Communication with the LTVerse ~

    Clearly, something needs to be done about my communication habits. Here's the good news: a large part of what's been causing me to not want to communicate with you all is, as mentioned above, the rather unglamorous nature of my work. I wanted to come back to you guys with guns blazing, yelling "YESSS LOOK AT THIS! AND THIS!! AND BETA IS COMING UP!" But I'm ready to accept that I can't work that way anymore. Showman Josh and Perfectionist Josh are mostly dead at this point, and I think you'll all be pleased to hear that Pragmatic Josh is now in full effect. At this point, my mentality on any given day is 'find the shortest path between where I am and LT being released. Follow it.'

    So from this point forward, I will plan to use my Sunday evenings to write a dev log -- however brief, unglamorous, or unexciting it may be -- to inform you all of my work over the past week. I'm not yet sure whether I want to do this every week or every other week, so I'll have to start with it and just go with what feels best.

    Again, I'm trying very hard to not make the same mistakes that I've been making with communication and end up disappointing you guys. To that end, I'm promising a regular but modest communication, something that will ensure you all that I'm still working to get LT in your hands without requiring as much of my time and effort as the daily dev logs, the monthly videos, RTB, etc. With FPLT I'm taking battleship-sized steps, but with communication we need to start back with fighter-sized steps :)

    (Oh, and yes, this does count for my first log...I'm not going to post more of my life story today :P)

~ Gratuitous FAQ / Autointerview ~

    I thought it might be interesting, if a bit strange, to ask myself questions from the perspective of forumers and squeeze some answers out of myself. In the event that someone actually does ask one of these questions, now I've already provided an 'official' answer. Maybe you will find it enlightening. Or perhaps just strange. Either way, enjoy :ghost:

Q. Thanks for joining us Josh.
A. Thanks for having me, imaginary forum member. What can I answer for you?

Q. Well, I read your post. I honestly did. But all I really heard was 'code code code C++ Python code Lua code JIT beep boop mainframe.' So...what's going on with Limit Theory, you know, that GAME that I'd like to be playing right now?
A. frustrates me too. I don't enjoy working on the technical side of it all nearly as much as I used to; frankly I would much rather be tying off loose ends on the AI, zone control mechanics, simulation LODing, unstable wormholes mechanics, etc. But, as I explained, this problem must be solved before I can finish LT, and I would rather throw myself entirely into solving it than split my time between solving it and trying to write gameplay code when the platform for said code isn't yet a solved problem.

Q. So it sounds like you've made no gameplay progress then.
A. Not true, but it is true that I haven't made nearly as much gameplay progress as you would want from two years of work (see above: >95% effort dedicated to the big problem).

Q. Oh? So then, what part of the game has changed the most since we last saw it?
A. AI has progressed quite dramatically. The top-level AI action ('Action_PlayGame', heh) is much more fleshed-out than before, meaning AI players are capable of literally 'playing' Limit Theory to a greater capacity than before. There are still missing elements, mostly because of missing logic for actions that deal with higher-level concepts like war, research, and construction. There's still a good bit to do here, but, having written some AI now in both Python and Lua, I've gotta say that, if we can get a fast scripting solution to work, AI development is going to go much faster when resumed full-time!

Q. Cool. You mentioned some stuff in the post that kind of scared me and made me think you're basically re-writing LT. Is that the case?
A. Definitely not. I'm changing the foundation on which LT rests, but by no means have I torn down the building. As stated above, the 'core' pieces of LT, i.e., much of the actual 'engine,' is still present in the C library. GPU code is all present and doesn't require porting. As for gameplay code / stuff that used to be in LTSL, I've ported it enough times now to know that it's a quick task. I've said it before and I'll say it again: it's the ideas beneath the code that the majority of the time to come up with. Typing code is easy. Coming up with an idea that will make the code do something wonderful isn't. Very little is 'lost' in this process.

Q. then, you've got it rendering and such?
A. Yep, I've got it rendering under both the current attack vector (LuaJIT) and the older one (Python). It's not really a big task with the library in place.

Q. Sweet, so, screenies?
A. Don't...even...!

Q. But I... :3
A. It looks the same as what you've already seen. I haven't worked on graphics! Seriously, the graphics monkey is in the cage. Gameplay, yes. FPLT, very yes. Graphics, no. I literally haven't changed the LT graphics in two years. It's been hard for me to control myself like that, but yes, I'm ready to accept that the current graphics are absolutely good enough for release. LT 1.0 is going to look very much like the latest screenshots you've seen, except with the missing content filled in. No more 'new metal shader version 298.0'.

Q. Fair enough. I'm glad you managed to control the monkey. Speaking of monkies, when beta?
A. When I solve FPLT and steamroll the remaining gameplay content.

Q. Ugh. I hate this 'FPLT' thing. You know, now that I think about it, what if you just ... ? Wouldn't that work?
A. (Depending on nature of ...): No, sorry. / It's a good idea but I've tried it and it didn't work out. / Well, that's essentially what I'm trying to do. / Dear God, why didn't I think of that, you're a genius, FPLT IS SOLVE \o/ *hands over bank account while weeping tears of joy*

Q. Okay then...back on-topic, why did you desert the forums?
A. I honestly didn't mean for it to become a 'thing.' It snowballed out of control: I just wasn't on the forums for a while, not for any particular reason, and then one day it was like a 'Josh is gone' thing, at which point I could have chosen to say "No I'm not," which would have pretty much done the trick. But instead I chose to give in to anxiety and was all "oh no, I'm gone!? Well now I have to come back with fireworks! :?" I really regret making that decision and letting it stagnate for so long.

Q. You should, a lot of us pretty much lost faith in you.
A. I understand. To be honest I can't say that I deserve more. All I would ask is that you give LT the benefit of the doubt in the sense that when it comes out, judge it as Limit Theory the game, not as Limit Theory the product of a guy who let you down.

Q. We'll see. What about Kickstarter though? I mean it's been even longer in the dark for them...
A. It's been a general pattern throughout development that I post more frequently to the forums than to KS. Historically, I've only put up KS posts when I felt that I had 'very substantial' material to post. As per above with the whole 'not a glamorous era in development,' I felt that I'd be letting everyone down with a text-based post, even if it's the best way to bring everyone up to speed. Once again, a regrettable choice made by Showman Josh, but now attempting to be set right by Pragmatic Josh.

Q. I like the sound of this 'pragmatic' Josh guy. Maybe he can finally get us a game?
A. Yes.

Q. Again, we'll see. I'm almost ready to dip my toes back into the water, but the truth is, you seem pretty good with words, yet have let us down quite a few times with those words.
A. Which is exactly why I've not sugarcoated this situation. Did you get the same 'thrill' reading all this that you did when I released an update video? Most assuredly not. But what I'm good at is raw honesty, which sometimes makes it hard for me when I honestly believe in my capability to do things at which I end up failing, or when the situation is honestly not so great. But I think you all, at this point, would much rather hear not-so-fun truths than nothing at all, which are the only two choices I've got. Notice the lack of glitter, unicorns, and grandiose promises in this whole thing!

Q. I certainly noticed the lack of glitter. Speaking of which, why on earth did you not provide a TL;DR for what may be your most obnoxious wall of text ever?
A. I considered it. I even wrote a TL;DR. But I realized very quickly that doing so was completely counter-productive to the point of this whole post. I took a lot of time to try to explain the current state of Limit Theory development; to try to get everyone to understand precisely what I'm doing from day-to-day right now and why it's necessary. If someone truly wants to understand it, they need to read this post. If not, then 'LT is not kill' should suffice. I can't explain it all and make my explanation super-compact. I could do that back when everything could be explained with images or videos. I can't do that at the moment.

Q. Indeed. Well, I'm rather exhausted from reading, and it's apparent that you're not going to give me screenies, so I'll end it here. Thanks for the answers, and good luck with LT. I look forward to that (semi)weekly log.
A. Thanks for listening, imaginary forum member.

<3 Josh
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
User avatar
Posts: 4211
Joined: Sun Oct 07, 2012 3:06 pm
Location: Baton Rouge, LA

Re: The State of Limit Theory Development, 2017 Edition!

Postby Dinosawer » Sun Jan 22, 2017 11:15 pm

So, some things:


I like pragmatic Josh better than anxiety Josh. :ghost:

As for performance, why not use multithreading? I know you thought thread communication would kill it, but that's frankly because you were thinking about threads wrong.
(speaking as someone with a master in astrophysical computer simulations)
iirc (bear with me, it's been a while) you were thinking of a thread per system and those threads would have to communicate to exchange info about each other, but that's not a good approach for simulations.
Can give more info if you're interested. :)
Last edited by Dinosawer on Mon Jan 23, 2017 1:24 am, edited 1 time in total.
Warning: do not ask about physics unless you really want to know about physics.
The LT IRC / Alternate link || The REKT Wiki || PUDDING
User avatar
Posts: 5487
Joined: Fri May 09, 2014 1:08 pm
Location: Belgium

Re: The State of Limit Theory Development, 2017 Edition!

Postby SeymourTheLlama » Sun Jan 22, 2017 11:20 pm


Thank you so much. Thanks for all the hard, thankless work you've done over the past 2 years. I'm no coder, but FPLT sounds like absolute hell. I would likely have called it intractable long ago in your stead. Thank you for sticking with it for so long.

I'm sorry that you felt like you couldn't trust this community to accept your return without pretty colors and fireworks, but I completely understand. That was what we were so excited about back in the day. However, no matter how the rest of the forum feels, it's certainly true of me that I'm just happy to hear from you. Your return has been greatly anticipated.

I wish I could give any advice on FPLT itself, but this is all definitely beyond my ken. I'm just a humble CE student bumbling through college. :)

Thank you as well for your promise of weekly updates. That could do wonders for stitching this community back together.

Only word of advice: if on some Sunday night you really don't have anything to share for some reason or another, just tell us exactly that. I'll be chill about it, if no one else is (which would surprise me). It would just be nice to hear from you.

Again, thanks! Good luck with FPLT! I'll probably be praying for you, if that's chill.

Ad Victoriam!

Live long and prosper.
User avatar
Posts: 50
Joined: Fri Dec 28, 2012 1:35 pm
Location: College

Re: The State of Limit Theory Development, 2017 Edition!

Postby Talvieno » Sun Jan 22, 2017 11:48 pm

Well done Josh! :) A lot longer than I would've expected, but well done. :clap: :squirrel: :thumbup:
User avatar
Community Manager
Posts: 7728
Joined: Fri Apr 11, 2014 6:50 pm
Location: North GA, USA

Re: The State of Limit Theory Development, 2017 Edition!

Postby DWMagus » Mon Jan 23, 2017 12:10 am

Dear god, if this is your interpretation "a few paragraphs" please don't ever write a book-length post. :ghost:

On a more realistic note;

Tackle the problem the same way any business would. Create a think-tank. Have a private meeting with a few coders on IRC that do coding/scripting pretty hardcore and just bounce ideas around. Even if no new ideas are brought forth, or nothing substantial comes from it, it gives a round-robin focused attempt at trying to find a better way to piece together A & B.
Early Spring - 1055: Well, I made it to Boatmurdered, and my initial impressions can be set forth in three words: What. The. F*ck.
User avatar
Posts: 5066
Joined: Wed Dec 05, 2012 3:25 pm
Location: Denver, CO, United States

Re: The State of Limit Theory Development, 2017 Edition!

Postby CptTrips » Mon Jan 23, 2017 12:20 am

Thank you very much Josh
Posts: 1
Joined: Tue Jun 02, 2015 3:32 am

Re: The State of Limit Theory Development, 2017 Edition!

Postby Flatfingers » Mon Jan 23, 2017 12:24 am

Josh, thanks for your post. Mostly, I'm glad you're OK, but I am also pleased to hear that Limit Theory is still in work.

Having read your comments carefully a few times and given them some thought, I have a few suggestions. As always, they're meant to be constructive, partly for the project but also for you personally as someone who deserves to enjoy your life. I hope you'll consider them, but of course they're just suggestions and you're entirely free to dismiss them.

1. I appreciated the detailed notes on trying to square the circle of performance and code size tractability. For a static set of requirements, that can be a Very Hard Problem. It sounds like that's the case for the current vision of Limit Theory.

2. This is going to sound "what if you just...?" but I'm going to say it, anyway, because it sounds like you're still hitting a wall.

As I've said before: If you encounter a problem that seems too hard to solve efficiently, consider that what you actually have -- and should be trying to solve -- is a meta-problem.

The way to deal with meta-problems is a) realize that you have a meta-problem, b) STOP beating your head against the current problem; step back, objectively list the goals and requirements of the next level up, and put all the goals and requirements at that level on the table, then c) ruthlessly assess whether one or more goals and requirements can be removed or replaced.

By doing that -- by focusing on discovering and solving the meta-problem -- you will often find that your intractable problem either goes away entirely (because you changed the goals/requirements that generated the problem), or has become a kind of problem that can be solved with existing methods.

Maybe you've already done this. Have you?

What are the goals and requirements that generate the difficult code size vs. performance problem? Do you know what those goals and requirements are? Have you listed them? Are you capable of the really hard task of changing or eliminating some features after four years of development? Can you accept a Limit Theory that is 90% of what you really want in order to have a working game in 2017 versus a 100% game that is still not done in 2019 or 2020?

Show me the smoldering corpse of Perfectionist Josh, please.

3. A weekly devlog sounds great. If scheduling your communication is what will help you stay in touch with your backers and fans, then please do that. If OTOH you will find it easier to maintain steady communications by dropping a short (and I do mean short) note every few days that averages out to once a week or so, then please do that.

For most people, who are not going to understand technical details, "heartbeat" counts more than content. The grognards will complain about wanting MOAR DETAIL because they like to complain. But you are not required to try to satisfy that complaint. A simple heartbeat note, every week or so, with a brief comment on the area of the code you're working on, is enough. Really.

Let me emphasize that:


Please print out the above statement in a suitably large typeface and pin it immediately above your main working computer.

Seriously, how can we help you understand that this is the truth, for both Kickstarter and the forums?

5. Let me rephrase that last sentence in its better form:

I sincerely ask you to please let us know what things we out here can do to help you succeed. For all the complaints, justified or not, you -- LT and you personally -- still have fans. If there's some way that we can help you, all you ever have to do is ask.

User avatar
Vice Admiral
Posts: 4413
Joined: Sat Nov 24, 2012 12:45 am

Re: The State of Limit Theory Development, 2017 Edition!

Postby 0111narwhalz » Mon Jan 23, 2017 12:44 am

JoshParnell wrote:Did you get the same 'thrill' reading all this that you did when I released an update video?

User avatar
Posts: 357
Joined: Mon Feb 01, 2016 11:40 pm

Re: The State of Limit Theory Development, 2017 Edition!

Postby BFett » Mon Jan 23, 2017 12:48 am

Thank you for the two years you have spent on this problem Josh. I appreciate this post very much and the continued work on trying to get LT out to the rest of us.

I hope those at your work and those here in the community are able to help you through this problem.

Best of luck Josh, keep it up!

PS: I agree with the below statement.
Flatfingers wrote:

Please print out the above statement in a suitably large typeface and pin it immediately above your main working computer.

Seriously, how can we help you understand that this is the truth, for both Kickstarter and the forums?
Last edited by BFett on Mon Jan 23, 2017 1:11 am, edited 1 time in total.
User avatar
Vice Admiral
Posts: 2395
Joined: Thu Jan 23, 2014 6:44 pm
Location: A galaxy far far away

Re: The State of Limit Theory Development, 2017 Edition!

Postby youngneil1 » Mon Jan 23, 2017 12:50 am

Thank you for the post, Josh.

It is much appreciated :) . What you do there is a fundamentally difficult thingie, so you can be proud of every bit of ground covered, even or actually especially if you had to traverse through a maze to make the next little step in the right direction. The most important aspect of growth here is ouside the current code base progress, but inside your ever growing abilities from coding and logical thinking to mustering the strength, courage and optimism to push through.

Will be here until you made it, regardless of when this will be :thumbup: :monkey: :squirrel: .
Posts: 26
Joined: Wed Feb 13, 2013 4:32 am

Re: The State of Limit Theory Development, 2017 Edition!

Postby charnode » Mon Jan 23, 2017 12:53 am

0111narwhalz wrote:
JoshParnell wrote:Did you get the same 'thrill' reading all this that you did when I released an update video?


Very much this. Josh, I appreciate the in depth look you gave us on your current progress and hardships. I do think, however, that you're misreading the community's expectation regarding desired information. Sure shiny stuff is nice, but I will always take bits of Information on the current state, with or without visuals. Backers went on an Adventure with you, not against you. It was very clear that there was a risk of Limit Theory never happening. As long as it is though, please keep talking to us, no matter however mundane you deem progress is. The suggested every other week rhythm would seem to be a nice middle ground to not distract you too much from working on Limit Theory.

Thanks for taking the time to lay it all out here.
You cannot not communicate.
Paul Watzlawick - 1st Axiom
User avatar
Lieutenant Commander
Posts: 191
Joined: Sat Dec 08, 2012 12:53 am
Location: Jenbach / Tirol / Austria

Re: The State of Limit Theory Development, 2017 Edition!

Postby p2k » Mon Jan 23, 2017 12:57 am

Hi Josh,

thank you for the insights of the current state of the LT development.
I hope that you will find your way to overcome the FPLT roadblock.

Thank you and good luck... :thumbup:
User avatar
Posts: 17
Joined: Tue Mar 26, 2013 4:49 pm

Re: The State of Limit Theory Development, 2017 Edition!

Postby kaeroku » Mon Jan 23, 2017 1:00 am

Josh you need to hear this.

All we need for regular communication is the second to last paragraph from your "solving the fundamental problem" section. We don't need the whole history. We don't need imagined interviews. We are asking for what is going on currently. Progress updates. Preferably monthly or weekly. That is ALL.

You take way to long to write these grandiose frakking monstrosities. Communication is not code. More doesn't solve the problem better.

You need to accept you are bad at communication. You need to either get good at it (which means, in part, identifying the most important things to say and limiting yourself to that, and doing it consistently,) or you need someone who can do it for you.

This post, at this point, does nothing for me because despite explaining what you've been up to I'm not any more convinced that "you'll do better this time" than I have been in the past. If you keep your promises about communication, that bay change, but dude you're on the thinnest of ice here in terms of community confidence, and your problem is not just lack of communication, it's knowing *how.*

I intend this as constructive criticism. Identify the weakness. Use that knowledge. I know Tal or some of the other mods would be happy to do a one or two hour voice chat with you every week and then pick out the relevant bits and share them with the community.

This is a problem which *has known solutions.* When you choose not to take known solutions to a problem, it shakes my confidence in your ability to apply the same concept to programming.

Please process this with as many cycles as it requires to fully compile.
Posts: 90
Joined: Sun Dec 09, 2012 4:02 am

Re: The State of Limit Theory Development, 2017 Edition!

Postby Dinosawer » Mon Jan 23, 2017 1:36 am

A bit more (aside from Y U NO multi-thread - seriously, Y U NO multi-thread) now I have more than 15 seconds to type:

Speaking for me personally, all you need to do to get my trust back is do what you promise to do.
I'll be frank, after the pattern of
"update soon! *crickets* update soon! *crickets* update this week! *5 months of crickets*",
I was surprised that you actually posted something at all this time. Very happily surprised, though.
Practical Josh and more-here Josh sound great, but I've read it before, and it didn't happen then. Please let it really happen now.
So basically, keep your promise of semiweekly devlogs and the only thing I'll have criticism about will be your coding efforts :ghost:

Aside from that, I strongly recommend making your next devlog about the actual gameplay stuff you did the last two years. Sure, it was only <5% of your time, but even that part of 2 years is bound to be interesting enough to write a couple paragraphs about, and some of us don't know a JIT when it flies in their face.
For example, I'm sure Victor will find little to relish in this post aside from practical Josh and not-dead Josh ;)
Warning: do not ask about physics unless you really want to know about physics.
The LT IRC / Alternate link || The REKT Wiki || PUDDING
User avatar
Posts: 5487
Joined: Fri May 09, 2014 1:08 pm
Location: Belgium

Re: The State of Limit Theory Development, 2017 Edition!

Postby mcsven » Mon Jan 23, 2017 1:42 am

Huzzah! Welcome back, glad to see you're alive and well(?).

FPLT sounds like a beast. I was vaguely amused reading your update because it sounds very similar to the process that Carmack went through when building Quake, and which Abrash recorded in his online book (relevant section here). He ended up writing multiple solutions to finally get the right one, exploring things in a practical way just to see what worked. He got there in the end so deffo keep banging that head against the wall!

Edit: having read Flat's post, I agree with this:

Fatfingers wrote:I sincerely ask you to please let us know what things we out here can do to help you succeed. For all the complaints, justified or not, you -- LT and you personally -- still have fans. If there's some way that we can help you, all you ever have to do is ask.

Please make the most of the community. It's a resource.
User avatar
Posts: 622
Joined: Sun Mar 03, 2013 5:41 am


Return to Announcements

Who is online

Users browsing this forum: Dinosawer and 6 guests