Return to “Dev Logs”

Post

[Josh] Monday, December 4, 2017

#1
Monday, December 4, 2017

Hey again everyone! Hope you all had a good Thanksgiving (for those to whom it is applicable). I certainly ate my fair share :ghost:

It's been a productive few weeks since the last log (minus Thanksgiving break), but last week, in particular, saw two major breakthroughs that have me really happy with the current state of affairs :)


Major Progress Since Last Time:

  • Major breakthrough with being able to use Lua data & functions from our C subsystems
  • Used the above to implement a C-side scheduler that schedules Lua logic for execution; significantly faster than previous implementation
  • Ported planets & atmospheres
  • Integrated Loom for advanced inspection of LuaJIT traces and assembly
  • Implemented several new algorithms for procedural geometry library, touched on in Lindsey's latest log
  • Loads of quality-of-life improvements to how 'game-level' Lua is written
  • First-pass design of mod format taking into account all existing support systems


We're All Modders: Hammering out LT Mod Format for Building Gameplay

At this point, we've essentially got all of our core support systems working. By that, I mean that we have either Lua-level or C-level (in some cases a mixture) mechanisms for supporting almost everything gameplay code could want to do: from defining new datatypes that can be used with the utmost efficiency, to adding and/or modifying existing ones, to writing gameplay code that needs to run in response to certain events, or needs to be scheduled to run at certain frequencies, to writing new methods that can be called on the aforementioned datatypes, to defining jobs that batch-process large chunks of entities that match a certain filter, and so on. Where there was once a wild-west of Lua trickery, there is now a solid set of production-ready, battle-tested system. So, what remains? Naught but the orchestration of said systems to compose the symphony of gameplay that is Limit Theory :)

So, it is time that we once again turn our attention to the format for this orchestration: the Limit Theory Mod Format. I have expressed many times that I wish for our vanilla LT gameplay-level logic to be written in exactly the same manner as any other mod, and I'd like to stand by that -- both for the sake of making it easier on us devs (since we'll have a well-defined, decentralized framework for incrementally adding modular bits to the game), as well as for the sake of ensuring that LT is not just moddable in theory, but is built on the very proof that it's moddable in practice.

Thus far, my solution is unfolding as a two-part mod format, where each part may exist zero or more times within any given mod file. One part is data, the other part is code. Data sections use a very simple 'data description language' to perform the aforementioned orchestration of our system. A data section for implementing an inventory might look like this:

Image

Hopefully it doesn't take too much brainpower to figure out what's going on here: we've opened up some object scope called 'Ship,' declared that it has an Inventory, and then gone on to define Inventory itself. Inside Inventory, we can see three types of 'declarations,' each of which corresponds concretely to a programming concept and each of which will be used in a different way to feed our underlying systems to build the required functionality.

As the programmers among you can likely guess, 'has' declares a field/member variable, 'can' declares a member function, and 'when' declares an event listener (a piece of code that will run in response to a particular 'event' that may occur on the object). The syntax of it all is just a sketch, so let's not concern ourselves too much with quibbling over whether 'has/can/when' is too cute / whether semicolons are the bane of mankind, etc :ghost:

The point is that we have a portion of the mod that's using a very simple 'data' language to inform all of our heavy-duty systems of how to make inventories work. The 'has' statements inform our CTypes system about fields, address, alignments, etc. The 'can' is informing our more-general type system about the presence of callable member functions. The 'when's are informing our event handler system about events and their listeners.

Then, of course, we must write some Lua to make it all work:

Image

And this looks more like what you would expect from standard Lua code. Here we're just implementing the functions referenced in the data section with standard Lua + some of the extra features added by the LT engine. Easy! All the 'bigger picture' elements that dictate when, where, and how a particular function runs are specified in the data section (e.g. addItem will be a method that can be called on any object with 'has Inventory', so that in other code we may say myShip:addItem(DenseTalvienite, 17) for example). Hence, much of the need for a modder to 'worry about' the internal mechanisms used to make LT performant (you know, the ones I talk about in pretty much every devlog ever?) is alleviated. Granted, they still exist and may still be interacted with at the code level, but for the majority of gameplay functionality, doing so should be unnecessary.

The Lua mod loader that parses all the mods into a format that can then be shipped off to the appropriate engine-level systems is already implemented; what remains is to do the shipping-off bit and to move existing gameplay code into isolated mod files where appropriate. I'd like to have this done by the end of this week :D

In response to anticipated concerns about 'oh no another language / LTSL / JOSH BE CAREFUL' :roll: : Rest assured, the goal is entirely different here. I didn't define a new scripting language; this is just a very simple data-description language that helps us connect all the moving pieces of our mod! It can be parsed with ~20 lines of Lua and the lovely string.gsub function. Anything more complex is unnecessary. I am not rabbitholing, I promise :)



Side-Soliloquy: Why So Much Concern for Moddability?

This bonus Josh-soliloquy is an optional part of your devlog experience :lol:

Spoiler:      SHOW
Many of my technical expositions, and even the less-technical ones, are ridden with talk of moddability and concern that a given system will not hinder the extensibility of LT. I feel that it might be a good time to step back and look at the big-picture of LT development for a moment; to express what modding means to me, what it has meant for Limit Theory's development process, and how it has impacted both to the utmost degree. To open, I'm going to make a daring assertion: I believe that Limit Theory would already have been released by this point had I chosen to stick with my original vision of 'no modding.' Take it as you will, perhaps noting that I do have a patently lackluster track record of timeline judgement (:oops:) Still, I feel that there is quite some truth to the statement. After all, we had the solid beginnings of a game with the release of the LT Prototype all the way back in April 2013 (!!)

Sometime in 2014, the tides began to change. I began to recognize the need for an external data format in which to place 'arbitrary' constants like those relating to high-level balancing (how much energy is required to negate one unit of damage compared to dealing one unit of damage? How much energy is required to extract one energy unit's worth of raw material from an asteroid? etc..) Naturally, it wasn't long before a similar need for 'arbitrary' code was recognized. The solution at the time was LTSL, and, as we all know, thus began the march toward the dark days, FPLT, and some of the hardest months of my life.

Looking back, it's clear that my solutions, from that point on, were guided by an unspoken requirement: LT had to be moddable, on that point I was no longer willing to yield -- else I would surely have eventually come back to C/C++, learned to write a more modular engine with faster compile times, and found a way to iterate on gameplay code without needing a separate language. It's hard to admit. But it's the truth. Hence, today, we are in a world where great sacrifices in development time and effort have been spent on Limit Theory's touted modding capabilities. Personally, I'm OK with that. And here's why.

When others ask about LT, moddability often comes up. Invariably, I bring up Limit Theory's suggestions sub-forums at some point, typically bragging on it as being "surely on of the best places on the internet to find inspiring, space-game/sci-fi ideas of remarkable breadth and depth, elaborated on by people of singular imagination and intellect." Since the KS closed at the end of 2012, there have been >1,000 topics and >20,000 posts in this subforum alone. It is a remarkable place -- one that I wish I had the time to read front-to-back. But that's just it: I don't. I don't even have enough time to read all of the beautiful, elaborately-thought-out gameplay suggestions generated by you all, much less implement them.

What I can do? What I'm really good at doing, as this whole process has taught me, and perhaps, what it is that I love doing above all else: it's probably not game design, it's probably not even procedural nebula algorithms :ghost: ...no...building things that allow other people to realize creation with ease, simplicity, and power...now that is the kind of thing that can get me out of bed in the morning. And that's exactly what I/we have done. We've built something that can take a high-level, simple description of a new gameplay mechanism, and do everything that needs to be done behind-the-scenes to let that mechanism play nicely with a thousand others, in the presences of tens of thousands of entities stretched accross a boundless universe. We've built something that renders many of your suggestions doable in a few hundred lines of Lua or less. And THAT...I'm proud of that :)

Everyone knows that Morrowind was one of my favorite games of all time. And it still is. It's still cranking, 15 years after release (in 2002). People still play it, people still buy it, people still love it. All because Bethesda decided they would build their game with a Construction Set rather than pure code, thereby enabling others to add to their original vision (which was already fantastic) via that tool. And the result is that today, some of us are still talking about how stunning Morrowind looks with graphics overhaul 3.0 et al; some of us are still in awe that Tamriel Rebuilt is alive-and-kicking, expanding the vanilla game's playable area by god-only-knows how many square miles.

And this is my dream for Limit Theory: that it ships as the vision you all supported in 2012 + the modding capabilities that have since become a core feature, and that, in 10 years' time, it has become something utterly and totally beyond that which I could have done myself or with our small team in the same amount of time. I really can't wait :)



I've left out all talk of a 'breakthrough' that happened last week involving Lua data playing nicely with our C systems. It's technical and I've decided to err on the side of not-so-technical this week, but, briefly, I'll just say that this breakthrough has opened up a lot of fantastic augmentations to our existing systems, as well as tremendous new flexibility in splitting data & logic between Lua and C (using whichever makes the most sense for the task at hand). Previously, Lua->C interop was easy. LuaJIT's FFI makes it basically effortless. But going the other way? Accessing Lua-side data / calling Lua-side functions from C? It was not so easy when that data/function could be arbitrary and needed to live alongside supporting C data (think: sparse logic scheduler). Since Lua is GC'd, we can't get pointers to Lua data, and this is a major annoyance about which I've ranted before. However, there's a function that I overlooked until last week...hidden in the Auxiliary Library API...luaL_ref...and it solves all my problems. *Slams face vigorously for having overlooked this gem.* At least I know now. :shifty:

Until the 'morrow! (Well, actually I plan to do my next log on Friday. If I'm 'close but not quite' with respect to the mod system, then it will be Monday. Stay tuned!)

:wave:

PS ~ Sorry for lack of screenies, but most of this work wasn't amenable to it. Besides, you guys just got a whole boatload of them in Lindsey's last log! ;)
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: [Josh] Monday, December 4, 2017

#6
Josh...




Will you marry me?









OK, fine, but will you hire me? Can I have like a dozen helpers to get all these ideas out of my head before it explodes? After you're filthy rich of course, so... July?

Not only does that data and code structure look like something I could easily understand and Manipulate, but...
What I can do? What I'm really good at doing, as this whole process has taught me, and perhaps, what it is that I love doing above all else: it's probably not game design, it's probably not even procedural nebula algorithms :ghost: ...no...building things that allow other people to realize creation with ease, simplicity, and power...now that is the kind of thing that can get me out of bed in the morning...

We've built something that can take a high-level, simple description of a new gameplay mechanism, and do everything that needs to be done behind-the-scenes to let that mechanism play nicely with a thousand others, in the presences of tens of thousands of entities stretched accross a boundless universe. We've built something that renders many of your suggestions doable in a few hundred lines of Lua or less. And THAT...I'm proud of that :)
This, right here, is exactly what I'm looking for. I'm genuinely overjoyed to hear you say this. I want to see the sort of mods that you make after limit theory, the kinds of mods other people on this forum make, the kind of mods I make. I want to see what people of all ages, people in Senegal, Sri Lanka, South Korea and South America do with this.

In the next few years, billions of people will come onto the internet for the first time, and I want them to enjoy, and contribute to, what I foresee to be countless communities built around letting anyone and everyone build their own games and worlds in the engine you developed.

That's a long way off, but from what you're saying here, it sounds a LOT closer to reality than if you had stuck with your original idea of non-moddability, releasing a good, but niche game.

Now, that said, consider this: Can I import external file types, perhaps convert them into something for LT, like a jpg, animation, sounds, etc? Can I create Probabilistic manually defined objects/scenarios/empires/etc. Which definitely exist somewhere in the infinite universe, but where exactly? The seed decides. Can I create the solar system, complete with imagery of the planetary surfaces? Can I create Asimov's Foundation? Can I create asteroid Tetris and have the npcs play it?
Image
When you're trying to fill an infinite multiverse, if you're not willing to consider the entire creative output of humanity as a starting point, you're wasting your time.
User: JoshParnell is accountable for this user's actions.
Post

Re: [Josh] Monday, December 4, 2017

#8
Hyperion wrote:
Mon Dec 04, 2017 8:29 pm
Now, that said, consider this: Can I import external file types, perhaps convert them into something for LT, like a jpg, animation, sounds, etc? Can I create Probabilistic manually defined objects/scenarios/empires/etc. Which definitely exist somewhere in the infinite universe, but where exactly? The seed decides. Can I create the solar system, complete with imagery of the planetary surfaces? Can I create Asimov's Foundation? Can I create asteroid Tetris and have the npcs play it?
In answer to this, I would say "almost certainly yes" to all of the above except animations, although the LT Tetris might be a bit weird. :P With most of the gameplay code in LUA, modding becomes quite simple, given enough hooks.
Have a question? Send me a PM! || People talking in IRC over the past two hours: Image
Image
Image
Post

Re: [Josh] Monday, December 4, 2017

#9
Hyperion wrote:
Mon Dec 04, 2017 8:29 pm
In the next few years, billions of people will come onto the internet for the first time, and I want them to enjoy, and contribute to, what I foresee to be countless communities built around letting anyone and everyone build their own games and worlds in the engine you developed.
He's not HYPE-rion for nothing :lol:


Talvieno wrote:
Tue Dec 05, 2017 7:38 am
Hyperion wrote:
Mon Dec 04, 2017 8:29 pm
Now, that said, consider this: Can I import external file types, perhaps convert them into something for LT, like a jpg, animation, sounds, etc? Can I create Probabilistic manually defined objects/scenarios/empires/etc. Which definitely exist somewhere in the infinite universe, but where exactly? The seed decides. Can I create the solar system, complete with imagery of the planetary surfaces? Can I create Asimov's Foundation? Can I create asteroid Tetris and have the npcs play it?
In answer to this, I would say "almost certainly yes" to all of the above except animations, although the LT Tetris might be a bit weird. :P With most of the gameplay code in LUA, modding becomes quite simple, given enough hooks.
External filetypes, almost certainly, I'll check with Josh. Animation has been a no-go since day 1. All of the other stuff, depends on how many hours you're willing to put in. So basically +1 to Tal's answer, except that I have to disagree about LT Tetris. We'd BETTER have some mods that are completely irrelevant to the core game :lol:
Blog | Twitter | Pinterest

"You’ve got to work on something dangerous. You have to work on something that makes you uncertain. Something that makes you doubt yourself... because it stimulates you to do things you haven’t done before. The whole thing is if you know where you’re going, you’ve gone, as the poet says. And that’s death."
- Stephen Sondheim
Post

Re: [Josh] Monday, December 4, 2017

#10
I am not massively familiar with lua, but it looks very simple. Im game. Beats Java.

Object Cat(){
Has Bool Hunger;

Can hascheeseburger;
}

Function hascheeseburger(self){
Self.Hunger == 0
Return "Yum"
}

^ yes, I know that doesn't actually work. I think I'm mixing python in? Ooh well, on a phone. Who cares

Edit:
Wait is this the part where you Github the project to only backers and use version control so that they can all contribute to end game gameplay code? Mods included in initial release? Keep the source C++ secret and just release the Lua?

Idk if Im making sense
Post

Re: [Josh] Monday, December 4, 2017

#11
LindseyReid wrote:
Tue Dec 05, 2017 1:29 pm
Hyperion wrote:
Mon Dec 04, 2017 8:29 pm
In the next few years, billions of people will come onto the internet for the first time, and I want them to enjoy, and contribute to, what I foresee to be countless communities built around letting anyone and everyone build their own games and worlds in the engine you developed.
He's not HYPE-rion for nothing :lol:
Even if wrong, I regret nothing. It's a worthy goal to aim for.


If you want REAL hype:
Spoiler:      SHOW
I'd love to see LT and what comes out of it to kill EA and avenge Will Wright and punish them what they did to Spore. :twisted: ... let alone their reprehensible behavior since. EA: Eventually Ashes
Image
When you're trying to fill an infinite multiverse, if you're not willing to consider the entire creative output of humanity as a starting point, you're wasting your time.
User: JoshParnell is accountable for this user's actions.
Post

Re: [Josh] Monday, December 4, 2017

#13
MyNameWuzTaken wrote:
Tue Dec 05, 2017 1:32 pm
Edit:
Wait is this the part where you Github the project to only backers and use version control so that they can all contribute to end game gameplay code? Mods included in initial release? Keep the source C++ secret and just release the Lua?

Idk if Im making sense
Nope. There aren't any plans for this to happen. :P
Have a question? Send me a PM! || People talking in IRC over the past two hours: Image
Image
Image
Post

Re: [Josh] Monday, December 4, 2017

#14
Talvieno wrote:
Tue Dec 05, 2017 3:05 pm
MyNameWuzTaken wrote:
Tue Dec 05, 2017 1:32 pm
Edit:
Wait is this the part where you Github the project to only backers and use version control so that they can all contribute to end game gameplay code? Mods included in initial release? Keep the source C++ secret and just release the Lua?

Idk if Im making sense
Nope. There aren't any plans for this to happen. :P
But using GitHub as mod backend would be very cool, easy to work together on mods or fork mods etc.

Or just use SpaceDock
procedurally generated comment
Post

Re: [Josh] Monday, December 4, 2017

#15
Miklos wrote:
Tue Dec 05, 2017 4:20 pm
Talvieno wrote:
Tue Dec 05, 2017 3:05 pm
MyNameWuzTaken wrote:
Tue Dec 05, 2017 1:32 pm
Edit:
Wait is this the part where you Github the project to only backers and use version control so that they can all contribute to end game gameplay code? Mods included in initial release? Keep the source C++ secret and just release the Lua?

Idk if Im making sense
Nope. There aren't any plans for this to happen. :P
But using GitHub as mod backend would be very cool, easy to work together on mods or fork mods etc.

Or just use SpaceDock
You are correct that the Lua will be released, and the engine and C++ will be secret; however, +1 to Tal's response: only Procedural Reality developers (ie me, Adam, and Josh) will contribute to 1.0 release code.

The community can use whatever tools y'all want for collaborating on mods with each other. ^^
Blog | Twitter | Pinterest

"You’ve got to work on something dangerous. You have to work on something that makes you uncertain. Something that makes you doubt yourself... because it stimulates you to do things you haven’t done before. The whole thing is if you know where you’re going, you’ve gone, as the poet says. And that’s death."
- Stephen Sondheim

Online Now

Users browsing this forum: LindseyReid and 2 guests

cron