Thinking about it, LT is still at the stage of secrets and whisperings in the dark corners of offices when it comes to development.

It must be killing Showman Josh.

Victor Tombs wrote:Thinking about it, LT is still at the stage of secrets and whisperings in the dark corners of offices when it comes to development.

It must be killing Showman Josh.

TOMBSTONED!

Say goodbye to EXTREMELY ambiguous messages and say hello to only partially ambiguous ones! :D:D:D:D

Victor Tombs wrote:Thinking about it, LT is still at the stage of secrets and whisperings in the dark corners of offices when it comes to development.

It must be killing Showman Josh.

LOL

First time I read that, I thought you said sNowman

FormalMoss wrote:LOL

First time I read that, I thought you said sNowman

Do you want to b--

NO.

JoshParnell wrote:OK, seeing a bit of angst so minor update time again:

Bet KS backers would also love such small updates ....

drLight wrote:

JoshParnell wrote:OK, seeing a bit of angst so minor update time again:

Bet KS backers would also love such small updates ....

I'm sure they would, drLight.

JoshParnell wrote:OK, seeing a bit of angst so minor update time again:

Last week brought several major breakthroughs that led to a huge leap in the completion of the mysterious platform upon which LT now lies. It is actually quite different than I expected, but everything that I've done, all the engineering, re-engineering, and various weird tech ideas that I've played with since the beginning seem to have somehow come together in a flurry and led me to the final result, and I'm really pleased and excited about it, both from a technical standpoint (yes, there's some very cool tech underlying that which makes the engine and game both performant and manageable), but also from a content authoring & gameplay standpoint. Fundamentally, it's EASY. Both engine and game work are easier even than writing LTSL was, but orders of magnitude more performant. Finally the tech investment pays off!

I think, when I reveal this final 'new LT' and the architecture underneath it, I will get some reactions followed by and then and, finally, ending in .

Best part is, I am now at the point where I can concurrently develop any pieces that I wish, be it engine, game, or the platform on which it all rests. So yes, game code is in the works concurrent to the finalization of the platform. It's pretty elegant how they can be developed concurrently, but in time you'll all get the details on that! I wouldn't have been capable of writing this solution years ago -- perhaps not even half a year ago, so, as strongly as I regret the fact that LT still isn't on you guys' hard drive, I think the timing happened to work very strongly to LT's advantage.

Good news is good news Still being stealthy, but man am I seeing the light lately.

Firstly: good to see things are still continuing.

Secondly: I'm surprised there's been no speculation on exactly what kind of funky tech Teh Josh has come up to enable him to concurrently code engine/content/platform? As far as I can see, the most likely candidate is some form of uber-LTSL which compiles to different targets. I'm not sure why that would be a real step forward though. What are the thoughts of the learned on here?

Even though I'm an OK coder, what Josh is doing is so far over my head I don't even try to speculate until he explains it.

mcsven wrote:What are the thoughts of the learned?

Define what you mean by learned, mcsven. I suspect you mean those who know about coding. Basically the code monkeys. Hmm...I obviously have a different definition of learned.

Edit: Was that too harsh?

Victor Tombs wrote:

mcsven wrote:What are the thoughts of the learned?

Define what you mean by learned, mcsven. I suspect you mean those who know about coding. Basically the code monkeys. Hmm...I obviously have a different definition of learned.

Edit: Was that too harsh?

I doubt so Victor, learned could mean anything, a high school grad woukd be more learned than a toddler, if you get my drift.

Personally, McSven is quite right in asking, what kind if tech in C/ assembler could Josh implement to get the performance tweaks he's suggested?
I remember Michael Abrash joined iD Software for Quake or Quake 2, to help optimise rendering pipelines back in the day.
Abrash now works with Valve last I heard.

So programmers/ techies/ anyone, what do you reckon Josh has done with this latest tech?
What's the mostly astoundingly difficult thing to achieve in a game engine that most programmers would never go near?

I think what has happened is he has followed the linux philosophy, and broken his engine into many parts, each doing their job independently of the others.

What I keep seeing is that he is making it easier to work on any individual part, and to try and avoid bugs propagating around when trying to fix them, which suggests to me that he has created a very modular breakdown of the old engine.
Something where each module takes a set of inputs and provides a set of outputs, where he can technically rip any of them out, and only break some functionality rather than all of it.
I expect that he has also optimized his compiling so that he doesn't have to recompile the entire engine but only the tiny part of it that needs to be redone (although modern compilers can do this even without a modular system).

But he was mentioning how he had this monolithic code, and how that was interfering with his work. So I expect he has broken it down into a less monolithic state.

While the end result shouldn't be that different to user's or modders, it should happen faster, as he no longer has to fight the entire engine to recode things. It should also provide a capability to replace chunks of the code without having to have the original source code too.

Although this is pure speculation, without knowing what the engine architecture looked like before we can't really make useful guesses.

mcsven wrote:I'm surprised there's been no speculation on exactly what kind of funky tech Teh Josh has come up to enable him to concurrently code engine/content/platform? As far as I can see, the most likely candidate is some form of uber-LTSL which compiles to different targets. I'm not sure why that would be a real step forward though. What are the thoughts of the learned on here?

I wouldn't call myself learned; I'm too often wrong about things here.

Still, if I had to speculate, what Josh described sounds to me most like the old programming dream of "black box coding."

This (for the curious) is the idea that when you're thinking about how to implement some behavior in software, what you want to do first is break down the entire system into a collection of smaller systems and define what those individual smaller systems must say to each other. This is pretty much what Silverware was describing.

Looking at code in this way is an example of Separation of Concerns, where you break down an intractably big problem into smaller ones that can be comprehended. From there came the idea of Structured Analysis and Design, which was a way to specify how components would connect to each other. (Dataflow diagrams were a product of this approach, eventually leading to the idea of dataflow programming.)

Computer-Aided Software Engineering (CASE) followed from the experiences with Structured Design, and tried to formalize those ideas into tools to help programmers build systems in this componentized way. That didn't last, but it occasionally comes back to life in simpler forms, such as Black-Box Programming.

And funnily enough, we had a discussion a while back here in the LT forums about this kind of componentization of functions. Back on October 22, 2013, Josh posted a devlog containing some of his thoughts about devising a programming language that's not based on parsing text. As he put it:

[color=#EE1111]JoshParnell[/color] wrote:Code and programs are such structured, beautiful things. But we write them using these unstructured sequences of characters in a text file. Then we write programs to interpret and create structure from these sequences. A bit perverse, isn't it?? We have this idea of some logical, structured piece of functionality that we want to create. And then we break it down into this horrendous, text-only language. We then suffer through typos, compiler errors, and all manners of bugs because of the fact that we're trying to communicate this elegant thing in an incongruent medium, and hoping that it comes out elegant on the other side after it's gone through the compiler (which is struggling just to make sense of it all). I'm really starting to think that it's all a bit crazy

What I really want to do is make a tool where code is built visually and structurally. Something like node-based code, where code becomes this simple, beautiful web of structure. Function and data would be the same thing - just nodes, and structure would be created visually rather than syntactically. Imagine a programming language where you never made typos, never had compiler errors, and seldom introduced bugs. I think that's totally possible if we re-frame the problem of building a program to properly reflect how simple and structured that task really is. Don't worry, I won't work on this until after LT, but I think I've found my next project after this one I'm thinking the ultimate test would be to write LT2 purely in that format, and see how simple I could get it to look if I really tried as hard as possible to express everything in purest, simplified form, unhindered by the restrictions of a text-based programming language.

We then had a brief but enjoyable discussion here in the forums about that idea of a visual programming language. (Prograph is one example of such a language system.) I wonder how closely the new version of Limit Theory, including LTSL, approaches this concept of a text-free programming language?

At any rate, why rebuild LT using some Josh-like form of this notion of structural componentization of functions? I'd say it's because doing so has several advantages for the programmer:

• It reduces the conceptual difficulty -- you can understand the whole system through its parts instead of all at once.
• By defining the inputs and outputs, you can test each component individually for correct behavior (unit testing).
• As long as you maintain each component's inputs and outputs, you can change its implementation however you like.

That last one, I suspect, is the main benefit that Josh is seeing from his latest rewrite. Having seen the big picture of what Limit Theory should do, he's been able to rethink how to define and organize the individual components in a more coherent way, yielding better performance, cleaner enhancement, and easier maintenance.

We can now start to see that, although almost certainly not planned this way, previous versions of Limit Theory were increasingly sophisticated prototypes:

• Generation 1: Hardcode the core behaviors. Identify which features are really required and how they might look and behave.
• Generation 2: Split the engine into core code and LTSL scripts. Analyze for performance, enhancement, and maintenance.
• Generation 3: Structural redesign based on Generations 1 and 2. Implement gameplay functions, test, and ship.

Seen in this light, the purpose of Generation 1 was to figure out what gameplay features LT really needs. The code was ugly and unmaintainable, but it was an important testbed for gameplay ideas.

Generation 2 was about figuring out how to implement those features. This was when we saw the existing LT modified heavily to turn it into an engine for processing LTSL scripts. It worked, and proved the value of LTSL, but it didn't perform well because it emerged from a previous structure that wasn't optimized from the start as a script-processing engine.

Generation 3 was built from the ground up as a script-processing engine. All the necessary functions for script processing, and for handling the relatively small number of gameplay functions that need to be part of the core, could now be identified and accurately described. This allowed them to be implemented in a consistent way, and (importantly) to make the connections between functions consistent. Less special-casing means code that's both faster and easier to enhance and maintain.

So in other words, I'm supposing that Josh (necessarily) did things the hard way a couple of times, but has now mostly rebuilt LT based on what he learned from those earlier passes. Ultimately that's really the only practical method by which finite and fallible human beings can craft software systems -- whether they're games or any other program -- that both have good performance and are (relatively) easily extended and maintained.

Again, then, while these are only guesses on my part, I think they're guesses that fit what Josh has showed us and told us over the years, along with being the hard-won experience of many working programmers.

As Fred Brooks put it: "The management question, therefore, is not whether to build a pilot system and throw it away. You will do that. […] Hence plan to throw one away; you will, anyhow."

Lots of good stuff there, gentlemen. What we need now is the man himself to break his link with the code for several minutes make his way to this thread and tell us how close the members who know about these things have managed to get to the actuality of what he's up to.

I wonder what the chances are of that happening sometime soon. And no Josh, this is not a demonstration of angst on my part.

It's obvious that members are interested in this subject. I can't see that you need to reveal any of your secrets in a reply...or am I missing something.

Personally, McSven is quite right in asking, what kind if tech in C/ assembler could Josh implement to get the performance tweaks he's suggested?

I think that is the more intriguing question. Others have offered nice guesses/assessments of what Josh might have been implementing, but all have mostly focused on the code design patterns, on the programming modularity. While certainly the code design can operate significant performance gains when it corrects previous design mistakes, firstly I doubt that a great coder always focused on optimization like Josh would have made such detrimental performance mistakes in the past.

Secondly, while beneficial to the productivity, I very much doubt that altering the code design, or the modularity alone, would give the orders of magnitude gain that Josh has talked about. I remember, however, Josh commenting that the use of scripting language within the main code has grown to the point of affecting performance, so I would guess that most significant parts of the performance gain came from better integrating LTSL within the main engine, or even from re-writing LTSL in a lighter-weight fashion.

In my experience, and from studying the experience of others, code redesign is the only reliable way to obtain orders-of-magnitude improvement in performance.

Optimization just does the same thing better. Code redesign does things a better way.