Return to “Dev Logs”

Post

Re: [Josh] Monday, November 6, 2017

#47
Love the update! Great job as usual!

This post's screenshots and knowing how powerful your procedural spacecraft generator already is really made me wish for animated spacecraft.
Like having some parts rotate/deform over time. Might be doable via rerunning the mesh generation starting at some iteration and slightly tweaking a variable.

Example spacecraft: Just an idea i'd love to see implemented in some way or another. :ghost:

Looking forward to playing the game. *<[:-)
Post

Re: [Josh] Monday, November 6, 2017

#48
I'm back from a break. Another excellent update: some tasty screenshots, and good news on progress toward straight gameplay feature/content coding.

After reading the scripting part of this update, though, I have a story to share, and a question.

This story concerns niceness, responsibility, and knowledge.

My coding career started on Motorola's 6809 processor-based Radio Shack Color Computer and IBM's mainframe running MVS. These systems were very different in power, but the experience of programming on them was similar in an important way: every program could be written as though it was the only thing running.

In the case of the CoCo, this was essentially true. Its OS was a single-tasking (albeit interrupt-driven) system. Once your code was given control of the CPU, it kept control until it was done (barring high-priority interrupts or I/O calls). As a programmer, you didn't have to worry about ceding CPU to other applications; there weren't any. So your code could just do whatever you wanted.

This was also true for code on IBM's System/370 mainframe... but a mainframe supported this "I believe I'm the only program running" perspective for multiple applications semi-simultaneously. It did this through "preemptive hardware multitasking." The OS would start your code and watch its behavior. After a certain amount of time, or if your code started an action that involved waiting -- like doing an I/O operation, which is very slow compared to straight CPU usage -- the OS itself would "preempt" your code. It would save all your state -- registers, stacks, the works -- then give the CPU to another process. Eventually the OS, according to its scheduling algorithms, would restore your program's state, and your code would get the CPU again until it completed (or the next preemption).

The important point here is that, in both of these cases, you as a programmer never needed to think about adding special breakpoints at appropriate locations in your code to explicitly yield control back to the CPU. You didn't need to know how to be nice to other applications. You just wrote your code as though the CPU was exclusively yours. Simple.

Then came Windows. It was a multitasking OS, but unlike a mainframe OS it didn't know how to do preemption all on its own. So now, in addition to making sure your code was correct, you also had to know how to structure your code so that you could explicitly yield control of the CPU back to Windows at appropriate places. Some programmers did this well. Some didn't. But everyone had to try, regardless of how this complicated the normal logic of what their code was trying to accomplish.

Then Windows and compilers and scripting languages got smarter, and once again programmers could write their code as though it's the only thing running, letting the OS take care of preemption for maximal efficiency.

And now there's the new LT scripting language, which again expects programmers to alter the natural structure of their code so that sleep(x) statements can be inserted in efficient places. This presumes that the people who want to mod LT with scripts will know how/where/when/why to inject a sleep(x) call.

None of this is intended in a critical tone. It would not be wise for me to criticize any technical solution Josh produces. :D If the best route to LT scripting that is powerful, performant, and relatively simple to support is to have scripters figure out where in their code they need to temporarily yield control back to the CPU, then that's just how it is.

I guess I'm wondering: would it be a good idea, if People Who Are Not Josh will be expected to shoulder that responsibility, to help them out by describing the rules for knowing where to put sleep(x) calls?
Post

Re: [Josh] Monday, November 6, 2017

#49
bdav wrote:
Fri Nov 10, 2017 1:15 am
Thanks Victor!
I have been following the logs for a long time, and finally decided to make a account since I had something to say :lol:
Welcome! Glad to see an old lurker make their first post :3

Josh says the short answer to your technical question is "no, it's not a problem". But I bothered him in the middle of coding to bring the question to his attention, LOL, so he may give a longer answer later :V
"You’ve got to work on something dangerous. You have to work on something that makes you uncertain. Something that makes you doubt yourself... because it stimulates you to do things you haven’t done before. The whole thing is if you know where you’re going, you’ve gone, as the poet says. And that’s death."
- Stephen Sondheim
Post

Re: [Josh] Monday, November 6, 2017

#50
Flatfingers wrote:
Fri Nov 10, 2017 12:46 pm
I guess I'm wondering: would it be a good idea, if People Who Are Not Josh will be expected to shoulder that responsibility, to help them out by describing the rules for knowing where to put sleep(x) calls?
I just asked Josh your question. Here's his response:
Actually Josh wrote:Naturally I will document how to do it -- making LT moddable has been a huge time & engineering cost, there is no way that I'm going to make it 'technically' but not 'practically' moddable...I respect my own time more than that xD
:)

Also, glad to see you posting again, Flat! Some of us were worried about you.
Have a question? Send me a PM! || People talking in IRC over the past two hours: Image
Image
Image
Post

Re: [Josh] Monday, November 6, 2017

#54
Flatfingers wrote:
Fri Nov 10, 2017 12:46 pm
And now there's the new LT scripting language, which again expects programmers to alter the natural structure of their code so that sleep(x) statements can be inserted in efficient places. This presumes that the people who want to mod LT with scripts will know how/where/when/why to inject a sleep(x) call.

None of this is intended in a critical tone. It would not be wise for me to criticize any technical solution Josh produces. :D If the best route to LT scripting that is powerful, performant, and relatively simple to support is to have scripters figure out where in their code they need to temporarily yield control back to the CPU, then that's just how it is.

I guess I'm wondering: would it be a good idea, if People Who Are Not Josh will be expected to shoulder that responsibility, to help them out by describing the rules for knowing where to put sleep(x) calls?
Trivially, it seems like any script that remains active for the duration of the game session should have at least one call to `sleep` in its loop. It appears that Lua allows for higher-order functions, so perhaps a modder could write a script for what happens in one iteration of their loop, and they pass it to a function called something like `LTLoop` which takes their function, and any arguments that their function needs, as arguments?

I don't know Lua syntax super well, but I think Python will illustrate my point well enough:

Code: Select all

def mod_loop_logic(*args):
    # loop statements go here

def mod_setup_logic(*args):
    # this is only executed once to set up the mod

# ...

# elsewhere, LTLoop is defined as something more or less like this:
def LTLoop(func, *args_for_func):
    dt = LTSleep()  # I'm going to assume we have some wrapper for `sleep` here that uses a sensible waiting interval and isn't very greedy
    while dt:
        func(*args_for_func)  # unpack the varargs
        dt = LTSleep()

# ...

# then the actual mod script could be something like
def mod_execute(foo, bar, baz. quux):
    mod_setup_logic(foo, bar)
    LTLoop(mod_loop_logic, baz, quux)
    return

Run your setup code once for loading the mod, then loop in such a way that it might not have the most optimized `dt` value, but it will avoid hogging the scheduler's time slices. This would mean that modders who don't have a great head for the scheduling aspect have a way to avoid worrying about the details, but someone more in touch with the metal can still write:

Code: Select all

def mod_execute(foo, bar, baz, quux):
    mod_setup_logic(foo, bar)
    dt = sleep(0)
    while dt:
        mod_loop_logic(baz, quux)
        dt = sleep(0)
And in our modder's handbook we just recommend "if you don't know what to do about scheduling and sleep calls, pass your function and its arguments to `LTLoop` and it'll take care of the rest".
Shameless Self-Promotion 0/ magenta 0/ Forum Rules & Game FAQ
Post

Re: [Josh] Monday, November 6, 2017

#55
Thanks, friends.

All is well; I just needed some time away.

Tal/Josh (and Grumblesaur), thank you for the specific responses. To people who've done it for a while, knowing where to scatter sleep() calls may seem so obvious as to not be worth mentioning. But others who've been given that responsibility may benefit from having the rules of thumb -- where you want to cede control, and where that's not needed, and why -- spelled out clearly. That information yields one fewer moment of enthusiasm-killing frustration when they're trying to turn a fun idea into working software.

The docs for the new LT script language sound like they're going to be extensive. If so, I hope all that work doesn't get deferred to the very end of the project. (Yes, that's Project Manager Flatfingers talking. :D )
Post

Re: [Josh] Monday, November 6, 2017

#56
Thanks all for the welcome :D
LindseyReid wrote:Josh says the short answer to your technical question is "no, it's not a problem". But I bothered him in the middle of coding to bring the question to his attention, LOL, so he may give a longer answer later :V
I guess it means that there won't be so many coroutines. Or that he already has a great solution, and it will come in a future log (one can hope!). Thanks for asking him :)

Flatfingers, it's true that this cooperative scheduling adds some constraints on the modders. But I still thinks it is done the right way for two reasons:
  • These are not programs, but scripts in a game engine. They are usually either event-driven or at fixed frequency, and in both cases they will naturally yield. As Josh said, there is a lot of sparsity in the execution. There may be some difficulties with scripts that need very long computations though.
  • All the coroutines are in the same process, sharing the same data. The cooperative threading acts like a synchronisation mechanism, but if you remove it (for instance by pre-empting), you need to add some other kind of synchronisation. I believe it would be more painful (and a waste of CPU) to have mutexes rather than yields. My point is that, in the cases that are difficult, the alternatives that I know of seem even worse.
Post

Re: [Josh] Monday, November 6, 2017

#57
Victor Tombs wrote:
Fri Nov 10, 2017 1:45 pm
Flatfingers wrote:
Fri Nov 10, 2017 12:46 pm
I'm back from a break.
:D Well, you know by now you were missed, Flat. You belong here and to not have you posting is a loss to the LT forums. Welcome back! :thumbup: :angel:
Happy to have you back Flat, but equally happy to know you do get away for breaks :)
:D

I hope you're well rested, and look forward to your sage opinions as we seem to be moving into a new stage of LT development.
:D :wave:

P.S. Josh, hopefully you know we love you, and we're so so happy with your progress and we're here for you if you need us :) :D
P.P.S If I don't speak for a few from the community, know that I'm so very happy to be a part of this family, even if I'm not around as much as I used to be, I am still here wooo :ghost:
YAY PYTHON \o/

In Josh We Trust
-=326.3827=-
Post

Re: [Josh] Monday, November 6, 2017

#58
bdav wrote:
Sat Nov 11, 2017 5:09 am
Flatfingers, it's true that this cooperative scheduling adds some constraints on the modders. But I still thinks it is done the right way for two reasons:
  • These are not programs, but scripts in a game engine. They are usually either event-driven or at fixed frequency, and in both cases they will naturally yield. As Josh said, there is a lot of sparsity in the execution. There may be some difficulties with scripts that need very long computations though.
  • All the coroutines are in the same process, sharing the same data. The cooperative threading acts like a synchronisation mechanism, but if you remove it (for instance by pre-empting), you need to add some other kind of synchronisation. I believe it would be more painful (and a waste of CPU) to have mutexes rather than yields. My point is that, in the cases that are difficult, the alternatives that I know of seem even worse.

To be a little more clear, I was not criticizing Josh's implementation.

It's a fact that not having to worry about when to manually cede control back to the scheduler eliminates errors and performance problems due to sub-optimal yielding. This makes life easier for programmers, who already have plenty enough sources of problems without having to worry about whether all their functions are being maximally nice to each other.

But it's also a fact that the value of a rule depends on the context of its application, and such is the case here. Josh appears to have concluded that the cost of expecting programmers to explicitly cede control in their LT scripts is outweighed (probably by a lot) by the benefits of 1) not having to write the LT core as a preemptive scheduler and 2) enabling scripts to exploit sparse activity to maximize the appearance of lots of "simultaneous" activity in the game world. In the context of Limit Theory and scripting, if Josh thinks this is the best way to go, it wouldn't even occur to me to assert that he's wrong.

My comments were not saying, "This is a faulty implementation." They were saying, "This puts more responsibility for performance on LT script writers, not all of whom will be experienced in manually ceding control, so I hope they'll be given some advice on how to do that."

Josh, via Talvieno, has answered that.

FormalMoss wrote:
Sat Nov 11, 2017 5:44 am
I hope you're well rested, and look forward to your sage opinions as we seem to be moving into a new stage of LT development.

/wave back

Rested, yes, thank you, and I make no claims for their quality but it's a safe bet there'll be opinions. :D

Online Now

Users browsing this forum: No registered users and 3 guests

cron