Monday, February 6, 2017

Logging the Creation of a Beast -- the Good, the Bad, and the Template Metaprogramming.

Re: Monday, February 6, 2017

Postby CSE » Mon Feb 06, 2017 2:48 pm

Disclaimer: no clues about what I speak about...

I do not understand what the computationally intensive logic is.
In my own words: I can imagine a very local logic that needs to run in real time, this is when e.g. many ships are involved in a dogfight and each ship's pilot AI need to decide what to do next. Add the real time sensors and the rendering, and I shiver. As this is real time, I understand it may be difficult to put it across threads, but it is also "standard" AI without very deep thinking.
Then there is the logic were the world evolve, and AI reacts to it, taking decisions and changing the world. This is not real time. Why can this not be threaded in a lower priority thread to be executed when the dogfighting is finished and the player either look at numbers on the market of a station, or look at the view while cruising or grind at an asteroid.

As I see it, this second logic contains the main world description, while the "real time module" contains a copy of the local environment only (happens infrequently, so the large data to transfer is not a problem). When landing/interacting with the rest of the universe, the background logic provides a new, updated, copy of the universe. If sometimes, when very heavy computations are required, there is a second of delay, it can be covered with nice landing graphics :?

So unless the problems lies already in the "dogfighting" real-time part (in which case you will have to simplify it), I do not understand where the show stopper for threading is.
Somebody cares to enlighten me?
Image
by Cha0zz
User avatar
CSE
Commander
 
Posts: 483
Joined: Thu Dec 27, 2012 3:59 pm
Location: Switzerland

Re: Monday, February 6, 2017

Postby Flatfingers » Mon Feb 06, 2017 2:49 pm

Thank you for keeping us (so to speak) in the loop, Josh.

This is exactly the kind of technical investigation that just plain takes time. It's not possible to guess or assume what some potential solution might do -- you have to actually build a version and go see if it works or not. And that does not happen instantaneously.

This is a tedious problem, but the space of possible technological solutions is way too big for it to be an impossible problem. Ad astra per aspera!
User avatar
Flatfingers
Vice Admiral
 
Posts: 4471
Joined: Sat Nov 24, 2012 12:45 am

Re: Monday, February 6, 2017

Postby SaberShip » Mon Feb 06, 2017 3:34 pm

Obviously a quantum computer is required. :ghost:

At any rate I am confident you will defeat this issue eventually. Keep attacking the problem from new angles and it is certain you will find something that is acceptable.
User avatar
SaberShip
Recruit
 
Posts: 1
Joined: Fri Aug 29, 2014 4:00 pm

Re: Monday, February 6, 2017

Postby TownEater » Mon Feb 06, 2017 3:59 pm

All that sounds pretty rough, and I can't think of any other workaround besides making your own JIT compiler but I feel that would just make things even MORE complicated. Good luck!
User avatar
TownEater
Recruit
 
Posts: 3
Joined: Tue Aug 12, 2014 3:50 am

Re: Monday, February 6, 2017

Postby afasthorse » Mon Feb 06, 2017 4:23 pm

JoshParnell wrote:Monday, February 6, 2017
'LuaThread' claims to be shared-memory and preemptive. Sadly it has mysteriously disappeared from the internet, and the only similarly-named impostors I can find do not implement true shared-memory/preemptive (likely because, as I found, it is not possible :ghost:)


Here's a github repository containing what I believe is The One True LuaThread: https://github.com/jjensen/luaplus51-al ... /luathread

Unfortunately it's no longer maintained and this repo is just holding the source of the original project, but maybe it gives you something to work from.

Alternatively you could try contacting the author directly, pretty sure this is him: http://w3.impa.br/~diego/

Good luck.
afasthorse
Recruit
 
Posts: 1
Joined: Mon Feb 06, 2017 4:14 pm

Re: Monday, February 6, 2017

Postby Dinosawer » Mon Feb 06, 2017 4:35 pm

JoshParnell wrote:lots of stuff

Looking back at some of my old tests, ~7% of time was spent in MPI communication, but those were sims with tens of thousands of particles, running at about 3FPS (though they were cleverly setup so not every particle was simulated every frame)
It's too late for me to figure out whether that's relevant. :ghost:
Warning: do not ask about physics unless you really want to know about physics.
The LT IRC / Alternate link || The REKT Wiki || PUDDING
Image
User avatar
Dinosawer
Admiral
 
Posts: 5608
Joined: Fri May 09, 2014 1:08 pm
Location: Belgium

Re: Monday, February 6, 2017

Postby Scytale » Mon Feb 06, 2017 7:50 pm

This was very interesting. If LuaJIT indeed fails to work out, do you have an idea of the next option to pursue? Or first things first?
User avatar
Scytale
Vice Admiral
 
Posts: 2794
Joined: Thu Nov 22, 2012 4:40 pm

Re: Monday, February 6, 2017

Postby 0111narwhalz » Mon Feb 06, 2017 10:28 pm

Scytale wrote:This was very interesting. If LuaJIT indeed fails to work out, do you have an idea of the next option to pursue? Or first things first?

He got quite excited when someone mentioned Terra, so I imagine that's on the JoshList.
User avatar
0111narwhalz
Commander
 
Posts: 396
Joined: Mon Feb 01, 2016 11:40 pm

Re: Monday, February 6, 2017

Postby Grumblesaur » Tue Feb 07, 2017 12:25 am

DWMagus wrote:Keep in mind the idea of threading on Linux before threading became 'mainstream' was to use sockets between threads. Sometimes you can even use streams. This way you're not really mixing data across the memory barrier, you're instead just writing to it like you would a file.

Not sure if this helps, but sometimes going backwards to see how things were done in the past might inspire ideas going forward.


So what we've learned today is that performance optimization with Lua is hard and using sockets is a dark art.
User avatar
Grumblesaur
Moderator
 
Posts: 1986
Joined: Sun Dec 02, 2012 8:57 pm
Location: Spotsylvania, VA, USA

Re: Monday, February 6, 2017

Postby Vartul » Tue Feb 07, 2017 1:44 am

afasthorse wrote:
JoshParnell wrote:Monday, February 6, 2017
'LuaThread' claims to be shared-memory and preemptive. Sadly it has mysteriously disappeared from the internet, and the only similarly-named impostors I can find do not implement true shared-memory/preemptive (likely because, as I found, it is not possible :ghost:)


Here's a github repository containing what I believe is The One True LuaThread: https://github.com/jjensen/luaplus51-al ... /luathread

Unfortunately it's no longer maintained and this repo is just holding the source of the original project, but maybe it gives you something to work from.

Alternatively you could try contacting the author directly, pretty sure this is him: http://w3.impa.br/~diego/

Good luck.


Quoting for higher visibility.
User avatar
Vartul
Captain
 
Posts: 583
Joined: Sun Dec 01, 2013 3:32 am

Re: Monday, February 6, 2017

Postby esteemdav » Tue Feb 07, 2017 2:54 am

Vartul wrote:
afasthorse wrote:
JoshParnell wrote:Monday, February 6, 2017
'LuaThread' claims to be shared-memory and preemptive. Sadly it has mysteriously disappeared from the internet, and the only similarly-named impostors I can find do not implement true shared-memory/preemptive (likely because, as I found, it is not possible :ghost:)


Here's a github repository containing what I believe is The One True LuaThread: https://github.com/jjensen/luaplus51-al ... /luathread

Unfortunately it's no longer maintained and this repo is just holding the source of the original project, but maybe it gives you something to work from.

Alternatively you could try contacting the author directly, pretty sure this is him: http://w3.impa.br/~diego/

Good luck.


Quoting for higher visibility.

seems fun, let me help
User avatar
esteemdav
Ensign
 
Posts: 19
Joined: Thu Sep 29, 2016 9:46 am

Re: Monday, February 6, 2017

Postby Freezin4aReason » Tue Feb 07, 2017 3:03 am

Thanks for the update. I enjoy reading the logs, though I understand very little of what was said since I've only dabbled in programming as a hobby. And that hasn't even been recently, but reading these updates makes me want to pick it back up again. :D

I did get distracted by this:
JoshParnell wrote:Monday, February 6, 2017

...I built a threadpool ...


Turns out you aren't the only one to build a Threadpool: :ghost: :ghost: :ghost:

Threadpool.jpg
Threadpool.jpg (132.23 KiB) Viewed 312 times


Anyway, good luck to ya Josh!
User avatar
Freezin4aReason
Ensign
 
Posts: 11
Joined: Sun May 31, 2015 5:04 pm

Re: Monday, February 6, 2017

Postby Dinosawer » Tue Feb 07, 2017 3:35 am

Might or might not be useful, but there exists and implementation for shared memory dictionaries in LUA (which seems to support LuaJIT too):
https://github.com/openresty/lua-nginx- ... shareddict

Might wanna take a look in their source and see how they did it, and expand it for tables/whatever you need to share
Warning: do not ask about physics unless you really want to know about physics.
The LT IRC / Alternate link || The REKT Wiki || PUDDING
Image
User avatar
Dinosawer
Admiral
 
Posts: 5608
Joined: Fri May 09, 2014 1:08 pm
Location: Belgium

Re: Monday, February 6, 2017

Postby DWMagus » Tue Feb 07, 2017 10:29 am

Freezin4aReason wrote:Thanks for the update. I enjoy reading the logs, though I understand very little of what was said since I've only dabbled in programming as a hobby. And that hasn't even been recently, but reading these updates makes me want to pick it back up again. :D

I did get distracted by this:
JoshParnell wrote:Monday, February 6, 2017

...I built a threadpool ...


Turns out you aren't the only one to build a Threadpool: :ghost: :ghost: :ghost:

Threadpool.jpg


Anyway, good luck to ya Josh!



Awwww.... I want one! :D
Image
Early Spring - 1055: Well, I made it to Boatmurdered, and my initial impressions can be set forth in three words: What. The. F*ck.
User avatar
DWMagus
Moderator
 
Posts: 5068
Joined: Wed Dec 05, 2012 3:25 pm
Location: Denver, CO, United States

Re: Monday, February 6, 2017

Postby kostuek » Tue Feb 07, 2017 11:36 am

I had a thought reading this update, so, just to add some noise... :ghost:

How much fps is actually needed for debugging? I mean, how about this:

1. Write a feature-complete version in Lua or in some other scripting language.
2. Bring all them nasty bugs down at 10 fps.
3. "Translate" all the Lua stuff into good old C/C++.
4. Bing down the few remaining bugs at 30 fps.
5. Be a happy human being.

Not sure about 2. or how efficient debugging at 10 fps or even less may be. Also not sure about 3. and the "translating" stuff. But hey, I tried :monkey:
kostuek
Ensign
 
Posts: 37
Joined: Mon Dec 22, 2014 6:17 am

PreviousNext

Return to Dev Logs



Who is online

Users browsing this forum: Talvieno and 7 guests