The Limit Theory Email/RSS News Thread

For LT discussion that doesn't fit into a more specific category.

The Limit Theory Email/RSS News Thread

Postby Talvieno » Wed Mar 09, 2016 4:54 pm

You can subscribe to this thread for future updates. This topic will remain locked but stay updated.
Link to Announcements RSS feed
You can subscribe to email notifications via a link at the bottom left of this page. This requires being logged in.

The posts in this thread are admin-created clones of Josh's original posts, and not posted by Josh himself.
The intention of this thread is to help people stay on top of news for the game even if they don't closely follow the forums. I'll add a date to each post as well as a link to the original. You can continue the conversation for the latest posts in their respective threads.
User avatar
Talvieno
Moderator
 
Posts: 6862
Joined: Fri Apr 11, 2014 6:50 pm
Location: North GA, USA

Re: The Limit Theory Email/RSS News Thread

Postby JoshParnell » Wed Mar 09, 2016 4:59 pm

November 16th, 2015

Kimny wrote:I don't know how many other different ways I can say I believe in him, I trust he is telling the truth, I trust his character, I like him, it's not a personal critique. Really, it's almost like it's necessary to spend paragraphs apologizing before making a criticism. And even I repeating all that in all posts, still one replies that I think Josh is lying, another replies saying I shouldn't distrust his character, another says I should shift to my personal life, you now reply as if I were trying to offend Josh.

I explained extensively in the other posts that on the contrary, I don't want to mix personal and professional. I really like Josh, trust him, etc, etc (second time in a post). I just think he has been in error in what regard not giving one piece of information during 1 whole year. Of course that is worth criticizing. And it's normal to criticize it. He is a professional, LT is a funded game. Sometimes it seems that taking 1 screenshot in an year, or even less, talking a couple of lines about the concrete game in an year, is to put super pressure or to ask Josh something very unusual. Indie developers certainly do not update on their games all the time as Josh did. Also as I said many times now, that's fine and I think he was very right in his change towards focusing in the game (I also have to repeat that all posts...). But it is not true that is normal and acceptable that indie developers go a full year without anything remotely concrete and behave as if it that was a plain fine decision instead of something to apologize for. Much less is usual that people are ok and even repeat in the developer's place that "progress is being made", that we should only thank Josh for everything nice he has (indeed) done so far or for not unplugging our critiques from a forum at the official page of the game!

It's very simple: he's been doing choices and following paths that deserve critics just as we would do with any developer. THAT is just what I meant by it's time to criticize Josh's approach to it. JUST that. But anyway, from now one I will reply with a signature that repeats automatically:I trust Josh's word, character and passion, I like him and admire him personally. Yet critique some of his decisions as I would do to other professionals (and that's a sign of respect). Btw, that's the third time self-justifing all that in the same post....


Hey Kimny, welcome :wave:

I like your mixture of respect and critique. I've no problem with it at all, and agree with you that it's been a rough and very-much criticism-worthy ride since last year. That's just the hard truth of it -- next time I make a game I'll be sure not to put myself into that position. But this time is difficult, I've made mistakes, had to go through mental issues, and had to spend a lot of time recovering from them and unraveling some of the poor decisions that I've made within the LT codebase during the time that I was losing my good judgement.

So the truth is, no, I don't have anything to show at the moment, at least not graphics wise. But there is a tremendously-good reason for that ;)

Spoilers!

Since people are getting angsty again, let me give some more detail on what I've been doing with LT. It's quite technical but very, very important to actually being able to finish the game. So remember how I used to have this concept of testing 'individual features' in sandboxes using LTSL? Right, well, that was all well and good. But at the end of the day, it actually wasn't all well and good, because, LTSL used the entirety of the game engine (liblt.so , liblt.dll, liblt.dylib depending on platform). It had no choice, because the game engine was one big blob. That means that, while each sandbox may only have called on particular features of the game engine, the entire engine was still there, in the background. Way too stuff was affecting the sandboxes simply because the engine wasn't separated, it was 'monolithic.' Changes to improve the functionality of one sandbox would end up cascading through the entire codebase and, far too often, breaking other functionality in other sandboxes. It was remarkably frustrating and time-consuming.

I've spent the past few months, in particular, completely changing this problem. LT is no longer a 'monolothic' engine but rather a compartmentalized ('micro') engine, comprised of many interlinked object files, each of which are independent from most others. Blah blah blah, technicalities. Why should you care? Why am I not able to show screenshots of this?

Here's the beauty: there are no screenshots of the game simulation, of the AI, of the core game logic, etc in isolation -- and yet, I can develop them in isolation because one does not need graphics to develop any of these critical pieces of the game!! 'Sandboxes' are now simple console programs that give me readouts of market data, faction stats, etc etc. They have no concept of graphics. OTOH, the asset generation algorithms, which are a heavy WIP at the moment (I hope to be able to show you all some output as soon as I finalize the structure of the algorithms), need graphics but no other concepts from the game. They are simple graphical sandboxes to view single models, just like Maya or something of the sort.

So. The game? Here's the deal: you won't actually see it until the component pieces are finished. At that point, we will wrap the player interaction layer over all the components, and that will be the LT alpha.

That's a high-level look at what's going on right now. Frankly, I'm really excited about it. On a day-to-day basis, it makes me a lot more comfortable to be able to work towards a specific goal without touching code that needn't be touched. In general, it makes me quite a bit more productive. It might sound like another one of my little 'perfectionist' tendencies, but, in fact, if one considers the complexity of a monolithic engine vs. a micro engine, roughly-speaking, it's the difference between quadratic O(n^2) time to develop and linear O(n) time -- that's because the separation of dependencies ensures that touching one piece of a micro engine doesn't cause a cascade effect that breaks all the other pieces. This is the epitome of the "wall of separation" concept that every good CS student should learn, but which I really failed to do well during my first attempt at structuring the LT engine.

"Ok ok ok very technical, much wow, BUT I STILL WANT TO SEE THE GAME AND STUFF." I know. Once I have asset generators in good shape, I'm going to stop being so stingy. I will be showing example assets as they come out of the pipeline when that pipeline is of sufficient quality. But, once again, as I have hinted at before, the next time you see a shot of the whole game (not just graphically, but including interaction and gameplay) will essentially be very, very close to BETA.

Did that make any sense?? I hope so! :geek:




I just want to add one more thought. For those of you who remember the original "Limit Theory" and where the name came from (the Theory of Nonexistence of Extrinsic Limitation -- scroll down to "The Meaning of Limit Theory"), it is actually critical that I implement the philosophy of Limit Theory in my work on Limit Theory. This game cannot be brute-force made by one person. It is only by the coalescing of a million tiny 'insights', the breaking down of a million tiny mental barriers, that a game like this can be feasible for a small team. Naturally, most of those insights are not graphical, but technical (or artistic-technical). We're all tired of hearing about the technical, but, at the end of this project, when Limit Theory is out and in your hands and you're making your way through a beautiful universe doing whatever you please, you'll have one and only thing to thank for that: those insights. Not a hundred artists, not a massive development team with a massive budget, but an engine and game built off of 3+ years of recursively searching for elegance in each piece of the game. When you are playing LT, you should thank the Limit Theory :)

The dev style is the embodiment of Limit Theory, and the release of LT is intended to be a 'proof' of sorts. The name was chosen as such from the beginning to reflect this concept :) Again, for those who haven't been with us since the beginning, go back and read the post I linked. It should tell you all you need to know about what's happening right now. I am searching for the final insights that I need :geek:


Link to original: viewtopic.php?f=11&t=4640&p=114850#p114850
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
User avatar
JoshParnell
Developer
 
Posts: 4080
Joined: Sun Oct 07, 2012 3:06 pm
Location: Baton Rouge, LA

Re: The Limit Theory Email/RSS News Thread

Postby JoshParnell » Wed Mar 09, 2016 5:03 pm

December 5th, 2015

tl;dr - I know what I'm doing.

Zanteogo wrote:It's just from this side of the computer screen I remember the years of LT development where every month massive improvements and changes were occurring and it seemed like you were on the highway to completion. It seemed like everything was coming together. I know that in almost all software, it's the way of things, the first 80% is the fastest, the last 20% takes 80% of the time. However, it seemed you had mostly side stepped this trend with your one man team approach, (for what ever reason). I recall the day when your showed us LT, and the AI was doing it's own thing and reacting in it's own way to changes, and I recall in awe thinking, "it's happening".

I know I am arm chair quarterbacking here, I am merely seeking to understand and to remove my ignorance to the whole thing.

However, to me, it was just before the start of LTSL that everything seemed to.. stop. Everything seemed to be put on hold. For the things that didn't, it's when the circular development seemed to really happen. Again, this is all from the prospective from this side of the computer screen of course.

From what you have opened up to us on your dark days, you admitted to getting stuck on circular cycle of not really moving forward. You also informed us that something you had done to LT had to be undone.

During the half of year of LTSL development it was sold as the thing that would cause a massive content explosion. I know your keeping your cards close, however, so I don't know what you have done since, but I am guessing there has been no explosion. If I am mistaken I and truly sorry.


Yes, those were back in the days when I was simply pumping out C++ as fast as I could. And there's nothing wrong with that. I could resume that approach today and have a much lesser form (and non-moddable) of Limit Theory out in a very reasonable timespan (and that's only because I've re-architectured the engine and now know how to prevent monoliths). But that's not going to happen, because, as my conception of Limit Theory matured, modding became something that I simply had to have. I want you all to be a part of Limit Theory's development. I want to see what people can do with this technology over which I have slaved for years. I want to play insane variations of my game that blow my mind. And it's all very much possible. I'm fine with dropping other content (especially content that has crept in since the original design doc) -- it can be appended later via modding. But I'm no longer fine with a non-moddable LT, and this is where the real challenge lies.

About LTSL -- the need for LTSL originally arose from a feeling of uncleanliness that I was getting from the LT codebase. It was taking me longer and longer to make trivial changes and recompile / view the outcome, it was getting harder and harder to figure out what was affecting what, and debugging was becoming more and more challenging. All of these problems were scaling linearly with the size of the codebase, meaning the difficulty of development was, effectively, scaling quadratically with time. This is, perhaps, responsible for how 'suddenly' things seemed to change. Hitting the wall was not a linear-time process, unfortunately. LTSL was nowhere near a mistake, and it did cause a fairly significant explosion in my eyes. A lot of it was UI, but that's because UI is one of the most annoying things in existence to hard-code. Remember the latest market interface? The assets interface with holographic views of each asset? There was also a working ship builder UI that was to be revealed in RTB 3.0, which never happened. Warp rails and their graphics effects, HUD, radar, target information UI, scanner UI, custom AI maneuvers, new station algorithms, new ship algorithms -- all LTSL. LTSL did the job. But it was a temporary 'escape' from the real problem: that the engine had grown too difficult to deal with, hence the need to 'escape' into a lightweight scripting engine.

But, as nothing is perfect, LTSL began to show its own problem -- performance. Towards the end of LTE (the Limit Theory Engine), much of the functionality was offloaded to LTSL in an attempt to fix the 'monolithic' problem. But LTSL struck back with performance limitations. It couldn't handle the intensity of true engine work. We were stuck. I could keep pushing and pushing and pushing in an engine that was becoming increasingly difficult to maintain, I could suck it up and accept the (fairly drastic) negative performance implications of engine work in LTSL, or I could use everything I had learned from both experiences and do something better. Anyone who knows me knows immediately which option I chose.



Here is what I would like everyone to fundamentally understand about Limit Theory:

It has ALWAYS been about working smart -- it has ALWAYS been about creating a beautiful technology in order to enable the creation of a beautiful game.

In the beginning, plenty of people said "no way, not gonna happen with a one-man dev team." And then guess what happened? I developed technology that allowed me to show the progress of a game at quite an impressive rate.

Despite the paradigm to which so many have become accustomed -- "shiny new content!" -- Limit Theory development has always been fundamentally about THINKING and finding the smart way to do things. That's the only way it can happen.

And that's exactly what's happening now. Yes, it has taken time, but I now have an elegant technical solution to both the monolithic engine problem AND the scripting performance problem. That much I will divulge. When this solution has been built up to the point of supporting all of LT's existing content, the content algorithms will simply be transferred over -- no work will have been in vain (not even LTE or LTSL, because they were important milestones for my coming to understand the right way). This time, I can guarantee that there will be no circularity, simply because there is nothing more that I could possibly do, technically, to energize LT's development than what I am doing now (to explain what I mean would be both too divulging and too technical). There will be plenty of back-and-forth in finessing the game, but not technology. One can either choose to believe or not believe me in this regard...it doesn't really matter, as LT will come in time and all of this will be over when the release drops.

^ This is the last time I will provide a justification of my current direction, I'll be keeping this link handy for the future.



And about that 'finding an entry point' thing that someone recently expressed (imagine my joy in being reminded of this dev log) -- it was never actually an issue. It was never about Limit Theory. What I was expressing in that log was a projection of my mental state onto my current task. The reality is that I was degenerating mentally to the point of being unable to see a big picture, to find a coherent strand of thinking to follow, to see LT as the beautiful concatenation of interesting subsystems that don't have an entry point. It is entirely possible to develop each independently, and to perform incremental testing by joining them as they are ready to be joined. I no longer have the problem that was being expressed in this log, because it was not a problem with Limit Theory -- it was a problem with me, and, as you all know, I've been working to fix 'me' since the end of the dark days.



PS ~ I would like to apologize for my tone. I realize it is unusual for you all to see anything but a dapper Josh. Current life circumstances + a change in medical circumstances + the tone of this thread have all summed up to a not-so-cheerful forum demeanor. For that I apologize.[/quote]

[quote="JoshParnell"]Thank you, Flat, for sticking with us and continuing to provide valuable input at every juncture. I hope you know that your words don't go unread by me, and, despite the points on which we may disagree, even in these late hours you continue to provide thoughtful-yet-non-destructive input on the dev process.

I'm sure many people think that their words fall upon deaf ears with respect to me (especially given how little time I have to actually respond to the plethora of discourse on here). Such is not the case (not since the Dark Days, at least). Criticism isn't going to cause me to tear down my whole development approach, but it certainly doesn't pass through my mind without leaving something behind.


Link to original: viewtopic.php?f=2&t=4859&start=180#p115867
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
User avatar
JoshParnell
Developer
 
Posts: 4080
Joined: Sun Oct 07, 2012 3:06 pm
Location: Baton Rouge, LA

Re: The Limit Theory Email/RSS News Thread

Postby JoshParnell » Wed Mar 09, 2016 5:05 pm

January 22nd, 2016

http://theadvocate.com/news/14649715-75/louisiana-technology-park-adding-video-gamers-iron-27-procedural-reality-as-tenants

Had to give a presentation to the 'board' today at work to be 'officially' accepted into the Tech Park (despite the fact that I've had an office and been working there for...a long time now...would have been pretty awkward had they say 'nah, get out' :ghost: ).

They were very pleased with what they saw :D Apparently several wanted to be on my advisory team, so I've now got some serious businessfolk who will be helping me structure the timeline / work / everything else as we get closer to release (I can almost feel the communal sigh of relief at the thought of Josh having some form of 'project management' team!) :)

Woo :thumbup:

PS ~ This article is actually how I learned that they voted 'yes' on Procedural Reality / LT. Hehe. Although I guess someone would have broken down my door and told me to get out had they voted no :monkey:


Link to original: viewtopic.php?p=118714#p118714
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
User avatar
JoshParnell
Developer
 
Posts: 4080
Joined: Sun Oct 07, 2012 3:06 pm
Location: Baton Rouge, LA

Re: The Limit Theory Email/RSS News Thread

Postby JoshParnell » Wed Mar 09, 2016 5:06 pm

February 24th, 2016

OK, seeing a bit of angst so minor update time again:

Last week brought several major breakthroughs that led to a huge leap in the completion of the mysterious platform upon which LT now lies. It is actually quite different than I expected, but everything that I've done, all the engineering, re-engineering, and various weird tech ideas that I've played with since the beginning seem to have somehow come together in a flurry and led me to the final result, and I'm really pleased and excited about it, both from a technical standpoint (yes, there's some very cool tech underlying that which makes the engine and game both performant and manageable), but also from a content authoring & gameplay standpoint. Fundamentally, it's EASY. Both engine and game work are easier even than writing LTSL was, but orders of magnitude more performant. Finally the tech investment pays off!

I think, when I reveal this final 'new LT' and the architecture underneath it, I will get some :wtf: :? reactions followed by :shock: and then :think: :o :o and, finally, ending in :clap: :thumbup: :D :squirrel: :squirrel: :squirrel: .

Best part is, I am now at the point where I can concurrently develop any pieces that I wish, be it engine, game, or the platform on which it all rests. So yes, game code is in the works concurrent to the finalization of the platform. It's pretty elegant how they can be developed concurrently, but in time you'll all get the details on that! I wouldn't have been capable of writing this solution years ago -- perhaps not even half a year ago, so, as strongly as I regret the fact that LT still isn't on you guys' hard drive, I think the timing happened to work very strongly to LT's advantage.

Good news is good news :) Still being stealthy, but man am I seeing the light lately.


Link to original: viewtopic.php?f=2&t=4859&p=120853#p120850
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
User avatar
JoshParnell
Developer
 
Posts: 4080
Joined: Sun Oct 07, 2012 3:06 pm
Location: Baton Rouge, LA

Re: The Limit Theory Email/RSS News Thread

Postby JoshParnell » Wed Mar 09, 2016 5:08 pm

March 9th, 2016

So yesterday I had written a rather lengthy response / update to this thread trying to clarify some of the questions popping up about the old vs. new underpinnings of LT...then I had to move locations, and when I hit submit my internet was gone and, naturally, phpBB doesn't save the form state so when I hit back my response was also gone (so I gave up). Gonna try again today, using the clipboard more frequently :ghost:

Here's a basic rundown of what was and what is, why we're in much better shape than before, etc :)

Old Setup:
  • Engine and most gameplay components written in C++ (~100+ source files since I like to keep things separated). Used some heavy-handed C++ constructs, adding to compile-time.
  • Despite being separated in terms of files, engine components were highly-interconnected, as I wasn't good at modular structuring when I started this journey.
  • LTSL implemented via compilation to expression tree (like an executable AST), scripts executed via this tree -- a fairly slow but clean approach
  • High build times due to interconnectivity and 'monolithic' structure (LT engine implemented in a single DLL / dylib / .so); even making changes to an isolated feature required an annoying amount of link time
  • Above problem led to higher use of LTSL for more and more features, eventually leading to exposing the performance limitations of LTSL...this ultimately backed me into a dark corner

New Setup:
  • Minimal 'platform' written in very light C++ (basically C; no usage of the 'heavy' C++ features, no usage of C++ standard library); platform contains core graphics / audio / system functions (i.e. the microcomponents of the engine)
  • Everything else, including higher-level pieces of the engine as well as gameplay, written in a specific subset of an existing high-level language (i.e., an industrial-strength one :P ). The platform configures, injects functionality into, and then starts up the language's environment. LT is then started in the HLL.
  • Wait! Engine functionality written in a high-level language??? Sounds like another performance disaster / LTSL situation waiting to happen! -- probably what you should be thinking right about now. This is where 'magic' happens. The platform on which everything rests allows for something rather awesome -- it allows for pieces of the HLL code to be transformed into native machine code that executes as fast as anything that was pre-compiled into the platform. The awesome part is that the code can remain written in the high-level language, it just needs to adhere to a few things, and mark itself as wanting to execute 'natively,' and the platform automatically takes care of the rest.
  • Ultimately, this means we have both performance-critical and non-performance-critical code written in the same language but executed in different ways. So the beauty of it all is that I don't need to guess whether or not some AI routine or PCG algo is going to be slow...if I run the game and it's slow, all I have to do is mark it (this is done in the language itself, via a mechanism I added), and next time it'll be converted to pure, optimized machine code that runs natively, just like all of the old LT used to run. There are a few drawbacks for code that wants to do this (namely, it can't use the full featureset of the HLL, otherwise it wouldn't be possible to convert it to low-level). But overall it's an awesome setup that makes development so much more tractable AND makes sure that we're ready to handle performance issues if and when they arise.
  • Due to this structuring, engine and game code can be developed independently. Entities needn't be rendered in 3D just to see basic information about what they're doing, what's in their inventory, etc. The old testbed approach is just magnified 100x here, because we can play with even more minimal pieces of the game thanks to not having a monolithic engine.

Please excuse my poor articulation of that all....I'm so caught up in the code right now that I think my english is degrading :P At any rate, the point is, this restructured foundation for LT solves both the performance issue and the development complexity issue (good lord, I can't even tell you how many lines of code are saved writing entity logic in the HLL!)

Yes, details are intentionally withheld until all is ready to be revealed :) For now, though, I hope some of you can take peace in knowing that the platform is finished enough to be able to work on both game and higher-level engine code. All three of these things can be worked on concurrently. And gameplay code is being written - far more quickly than used to be possible :squirrel: It is currently first priority, as I want to know as soon as possible which pieces of gameplay code will require the platform's 'magic,' and then test to verify that it does indeed remove performance bottlenecks as I expect (naturally this testing has already been done (successfully) on typical performance-benchmarking code, just not with LT gameplay code yet). Having written a good bit it already, I can tell you guys that, all this technical stuff aside, the single best part of this new approach is that building LT actually feels like building LT, not like thinking about the best way to implement each tiny piece of it. In that respect, a HUGE burden has been lifted from both myself as well as future modders!

I told you all I was going to play it stealthy until I'm ready to show everything, but there's a little peak at the structure so that you all don't have to guess as much (although now I'm sure I've left new things to guess at...) :ghost:


Link to original: viewtopic.php?f=2&t=4859&p=121865#p121861
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
User avatar
JoshParnell
Developer
 
Posts: 4080
Joined: Sun Oct 07, 2012 3:06 pm
Location: Baton Rouge, LA

Re: The Limit Theory Email/RSS News Thread

Postby JoshParnell » Tue Apr 05, 2016 8:54 am

April 5th, 2016

Hyperion wrote:It's been itching at me for some time now, as it probably has for plenty of others. I'll put it simply.

I want proof that development is taking place, that the game is closer to being finished than it was last year.

Not "Things are exciting" or "I can work so much faster now" or "I know what I'm doing" Not words and rumors about release date leaks, but cold hard proof. An image or video of something new, something we haven't seen before. It doesn't have to be much, but unless I missed something, it has been over 14 months since there was a single shred of proof that the game has developed beyond where it was in January of 2015.

Josh, I'm not calling you a liar, and I understand that you're holding your cards tight which is fine, but absolutely nothing since you declared the beginning of Golden Days almost 8 months ago? I'm sorry, but I'm not big on faith, and this total silence is testing those limits. Words and promises won't keep that alive, only evidence would carry me months and miles longer, and I have a feeling I'm not alone.

Fair enough, I can understand all viewpoints expressed in the thread so far (with the exception of TGS's 'nil' expectations for this game -- come on now!)

No game or engine screenies -- I'm still not graphically back to where we were previously (meaning I haven't brought all of the rendering and shader tech over to the new platform yet)...have actually been focusing a lot more on the gameplay side of things, and can I just say (I think I've already said this)...it's an absolute joy writing gameplay code in augmented Python (yes, this was going to be revealed in the 'proof' anyway) as opposed to C++.

I use dropbox for my version control (wait, what?). Yeah, seriously, consider it: I have like 4 different machines at the moment, and I need code to be current across all of them. At the same time I need the ability to rollback. Dropbox makes both easy (you pay an extra $15 or something like that per year to get an entire year's worth of version history regardless of how many versions there are or how big the file is....that's kind of amazing). The moment I save a file it's as though I've committed a new version, which is immediately available on other machines. Yes, this is awesome for a solo developer. No, don't try this on a team or many people will be angry.

So for your 'proof' I just printed out two simple version histories, one for ltheory.py (the entrypoint that basically launches the game without doing much else) and one for pyengine.cpp (please DON'T start using the name 'PyEngine' to describe what LT runs on because that's a temporary name!!) which is the entrypoint for the 'compiled' part, which, as I've explained before, basically configures and sets up what you now know is a Python interpreter, and injects facilities for graphics programming as well as JIT magic for pulling any of the performance-critical python down to native. Those histories, or rather, a few pages of them -- dropbox does a scrolling load thing -- are attached. Incidentally I did edit them both earlier today, despite the fact that these two files don't change much in comparison to the directory structures (hence the gaps). Most of what I do now is adding components (all of which are part of a module), data structures that are part of gameplay, shaders, etc. So most of my change is actually adding to the directory structures (no, sorry, not going to give a dir tree! Too many secrets :) ). In some places I actually have code that automatically scans a certain directory to load assets (this makes both dev and modding easier! For example, to create a new component that can be added to an in-game object and give it some kind of behavior, all you have to do is write a new code file (yes, .py) in the component directory and, if you did things correctly, you'll be able to call object.add(MyNewComponent(blah, blah)) without changing code anywhere else!

I know a few of you will now undoubtedly have reactions to the fact that I've just revealed the game language to be Python, (yes, I'm sorry Flatfingers...I am...), but please keep in mind
  • Huge, huge, huge amounts of thought went into choosing it over all the other options I had on the table, and after a lot of going back and forth, pro's and con'sing, running different tests, I was able to make this choice with full confidence.
  • Performance-critical code will not run in Python, so restrain the knee-jerk "oh noes it's going to be slowwwww" reactions (I mean, I would have that reaction if a dev said he was using python for a serious game...until he mentioned native JITing :) )
  • The 'magic' mechanism is of my own coding, this is not pypy or psycho or Cython, this is Josh + CPython. I chose this route because I believe it represents the widest spectrum of performance-devtime trading capability. None of the above, with the exception of psycho (which is now abandoned), could routinely perform as well as native code (Cython comes closest but it doesn't have as much runtime information to work with as I do :geek:). On the other hand, since it's CPython, it's a full and complete implementation of Python 2.7 (yes, 2.7, not 3.*, another calculated decision), meaning no restrictions or quirks. So in your python functions, you've got as much 'expressiveness' as you can possibly have. In your python functions that are marked for native conversion, you've got as much speed as you can possibly have (although you do lose a bit of expressiveness, naturally).

Yes, I was waiting to drop the P-bomb because I knew I would need to provide some comforting words with it :P So, those of you who plan on modding LT can run off and start learning Python (2.7!) now :)

And those of you who want proof that I work on LT, check the pdfs. If those aren't enough (because it's definitely worth my time to forge version histories?? :ghost: ), hang tight. I will try to get graphical 'proof' of the underlying progress as soon as I can :thumbup: Once again, I really appreciate you guys being patient with me. It's been a longer journey than I ever imagined, especially in terms of my own state, but it is converging :)

ltheory version history.pdf
pyengine version history.pdf



Link to original: viewtopic.php?f=2&t=5060&start=15#p123136
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
User avatar
JoshParnell
Developer
 
Posts: 4080
Joined: Sun Oct 07, 2012 3:06 pm
Location: Baton Rouge, LA

The Limit Theory Email/RSS News Thread

Postby JoshParnell » Thu Apr 28, 2016 10:11 am

April 27th, 2016

Hyperion wrote:Module instancing is actually proving to be a slightly difficult problem to hash out. Getting a good way to track 10,000 eventual variations on a single blueprint for every single blueprint without making the system cry is probably doable, but is beyond my realm of expertise. Just as a brief outline of my (probably dumb) thoughts, because this isnt really the place for it:

Blueprints are folders, but also instructions coded with the particular Tech stats. Inside the folder is a registry of every unique instance yet made and its stats, each tagged with the owner and where it's used. The owner and usage tags change freely via trade & plunder.

The currently simulated blueprints are known scripts, executed as necessary, changing the stats of the given instance.

Players & NPCs can also execute their own scripts (aftermarket modifications, improved/shoddy construction techniques & materials, overclocking, etc)on a particular instance which also changes the instance stats, but are wholly different from blueprint scripts. Such modifications don't affect the registry serial, but can affect the Human-readable name.

Destroyed instances (or intact but attached to destroyed ships/stations) continue to exist in the registry for potential salvage and reactivation until cleanup.

Temporary instances can also be attached with semi-random stats to permanent debris fields, but disappear as soon as a player is out of range.

Is this efficient? No idea :lol:


I think you've got most of the general idea, but I'll explain the actual implementation anyway (and why it is efficient). A blueprint is a pointer to another blueprint (its 'parent' or parent directory in your model), plus a list of attribute operations (you can think of that as a very, very limited script, but really it's better thought of as a 'diff' between the parent blueprint and itself -- an object-attribute-space differential, so-to-speak). This is highly memory efficient, as we require very little memory to make lots and lots of slight variations of an object. If the variation is slight (for example, a delta in just a single attribute), then the memory required for the BP is very, very small (quick mental calculation says ~20 bytes on 32-bit machine, ~32 bytes on 64-bit). "Absolute Original" blueprints (i.e., those that haven't been derived through researching a stem of an existing blueprint or implicitly derived by modifying an object (note that virtual blueprints are not visible to the player, but rather an abstract BP that exists solely to help the AI understand and quantify the nature of an object that's been modified -- by keeping an up-to-date virtual blueprint for the object, we allow the AI to apply the same reasoning to all blueprints when it comes to any type of analysis)) are simply blueprints whose parent field is 'None' and whose attribute operations consist of setting all attributes to their correct values (in other words, they're a diff with a 'null object', so to speak). These are significantly bigger than derived BPs, especially when they are for objects that have a 3D representation (because it is the base BP that actually holds the model). This paradigm of using chains-of-diffs pops up frequently in LT. It's a really, really good way to save memory when we're in procedural land where there's a huge amount of generated content, but where that content often has significant underlying similarities that can be captured in this original + diff + diff + ... manner. Kind of like prime factorization :geek:

So the key here is to think of a blueprint as a self-recursive object, much like a tree (except that it's really just a backwards-linked-list). As far as who's got them and when we need to delete them, this is of no concern: when a blueprint is lost -- let's say a ship carrying a blueprint that has no other copies in existence1 blows up and RNGesus decides the BP didn't survive, hence is nowhere to be found in the debris -- the refcount goes to zero and it's gone from memory. Reference counting really does work beautifully. Same applies to any temporary instances that might be generated -- as soon as no object is holding a reference to it (because, for example, the debris container whose inventory held the reference to this blueprint died of old-age and vanished from memory), it vanishes. Note that this hypothetical situation of finding a blueprint in a debris field may not even happen, since we try to always have an explanation for things existing rather than just 'generating them.' Then again, fields are natural sources of value within the game as they have a rate of mineral replenishment, so I could explore the concept of extending and generalizing that to different fields having their value source exist in different forms. :think: For example, a debris field might have a certain rate of 'value replenishment,' meaning once enough time has passed and the field is not at max value capacity, it may spawn an item of value <= its wallet, so to speak, and deduct that value from its wallet. In this way, although we spawn things, we still retain total analytic control over the injection of value into the universe. Doesn't seem too off-track from our motto of real gameplay dynamics, since we already do this with minerals. I might, however, apply a nonlinear valuation function to 'artificial' objects in this case, such that it's generally easier for a field to spawn a natural object or a very common (cheap) artificial object, and disproportionately hard to spawn really valuable artificial objects (I'll probably end up making it a power function, and throw the power into the universal constants so that the user can tweak it for a different gameplay experience -- that's what I do with most things related to valuation and value transfer, since they're somewhat 'arbitrary' compared to many of the functions and formulae in the game). An actual original blueprint is both artificial and typically of very high value, so I wouldn't expect to stumble over them often, even if you're a seasoned explorer who's seen a lot of fields.

Dear lord, I think I've just unified field resource regeneration mechanics with project manager capital expenditure mechanics :wtf: So really, an asteroid/ice/debris/whatever field is a project manager for a sub-component of the 'universal project' -- it gets a steady drip of credits flowing into its wallet, courtesy of the universe, (until said wallet reaches a cap, same as in the auto-manage component of the projects mechanic), and expends credits as it sees fit, depending on the type of field, to bring new resources into the universe. Think of that when you approach an asteroid field in LT...he's actually a high-pay-grade manager! :lol:

What am I doing. How did all of this start from your post about module instancing. I must be in the theorycrafting zone right now :monkey:

*opens vim and wanders off further into gameplay theory lala land*




1 Technically, to truly be gone from memory, all existing copies of the BP must be destroyed AND all objects built from it, AND the BP must have no surviving children (all BPs that were ever derived from it, if any, have been wiped from existence as well). So it's a very unusual event for a highly-utilized BP to be wiped from memory, since it will often have objects in existence that were built from it, hence carry a reference to it, and will often have child BPs that were derived from it (who will often have objects in existence that were built from them)...and so on. But that's exactly the behavior one would expect. It all 'just works' in refcounting :) An remember, BPs are very small memory-wise. I won't do the exact calculation, but a single planet's texture takes the same memory as roughly a bajillion blueprints.

Ahhh what is wrong with me why is this post so long :shock: :ghost:


Link to original: viewtopic.php?f=2&t=5060&p=124867#p124867
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
User avatar
JoshParnell
Developer
 
Posts: 4080
Joined: Sun Oct 07, 2012 3:06 pm
Location: Baton Rouge, LA

The Limit Theory Email/RSS News Thread

Postby JoshParnell » Fri Aug 12, 2016 12:53 pm

August 12th, 2016

Yikes, a few things certainly need to be said:

1. I've not abandoned you all. You may not see me, but I am with you! By that, I mean, I'm in this office working on LT.

2. I sincerely apologize if anyone feels that they've gotten the middle finger from me. Not my intention, ever.

3. Didn't pull a fast one on you all -- I know I'm going to have to keep saying this until the substantial update that you're all waiting for (good lord, with the amount of tension that's going to be relieved, I should start a poll to vote on a name for it...it needs some kind of epic, cathartic name that will go down in LT history) -- but there's really no option for me other than to finish LT. Don't take that the wrong way...I want to finish LT, I need to finish LT because it means so much to me and, after all, if you give up on your dream, then what do you have left? For me it would be devastating. Not to mention the devastation of letting you all down, which I already feel quite sharply due to how far overdue the game is. But it would be unbearably sharp were I to simply call it quits. And of course, it would be downright stupid, considering how far we've come, how many hours, how much energy has been invested -- there's no turning back!! Sometimes I really do wonder how anyone can think that I would actually consider that :monkey:

So, yes, these threads I imagine will continue until I present the next installment of the 'State of the Limit Theory' address. But said address is getting closer each day. Sorry for the angst, I guess it's worse for the people who don't know the relative closeness :shifty:

reegs87 wrote:I do hope that all is well with Josh. I would imagine that the guy has a tremendous amount of pressure on him. That being said it would be lovely to see an update sometime in the near future; this forum was one of my regular haunts....


Thanks, appreciate the sentiment. All is well, although it's been a bit busier than anticipated, but such is life. Recap of life in the past few months:
1. Dad sold house, had to pack up & move all my stuff
2. Femnode got a job in CA, took a trip out there to apartment hunt with her
3. Found and secured a new place to stay here (5 minutes away from the office!! LT Headquarters is just a super short drive away!)
4. Moved and unpacked all my stuff at said new place

But once more I seem to be in a stable life situation and am in the office every day cranking out the LT (and drawing up plans for how to repair the long-since-derailed hype train :ghost: )

kostuek wrote:I'm afraid, he may have hit the same wall as he did with LTSL - his Python business may still be too slow. If this is the case, he is probably back in his corner, thinking about what the hell to do now.


Thankfully not, so far LTEngine 2.0 (pff, I know, there's been a lotttt of refactoring over the years, but this is the first time I actually split the codebase) has been a joy to work in. I'd be lying if I said I wasn't anxious in the slightest about what kind of performance we'll see when I have all the logic running. But, that's kind of the nice upshot of having a hybrid engine -- if performance is problematic, you tell more of the code to come to the dark side...eh, the statically-compiled side. So far, though, all is well and only going to get better as I continue to improve the JIT engine to be able to handle harder cases (hence, be able to convert more of the code).

(Yes, this monkey is entirely aware that he did not give any specific details on the 'State of Limit Theory.' Don't read into it, because unless you read it in a positive way, you've read incorrectly :roll:)

PS ~ In case anyone considers me a lord/savior, which I hope is not the case, please cease said consideration...as we have all seen, I am, regrettably, quite mortal and flawed indeed :oops: However, this mortal still promises to bring you Limit Theory :thumbup: (remember my motto: "I'll ship it or I'll die trying!")

Link to original: viewtopic.php?f=2&t=5356&start=15#p132283

Talvieno's note: I feel like it's important to post this here, given how quiet Josh has been lately, so that no one thinks Josh is missing again or not working on the game. Normally I wouldn't post anything here that had so little info on the state of Limit Theory, but this time I'm making an exception.
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
User avatar
JoshParnell
Developer
 
Posts: 4080
Joined: Sun Oct 07, 2012 3:06 pm
Location: Baton Rouge, LA


Return to General



Who is online

Users browsing this forum: Dromeda5, Sarif, Yahoo [Bot] and 5 guests