Return to “Dev Logs”

Post

Tuesday, February 28, 2017

#1
Tuesday, February 28, 2017

A rather solid but unremarkable week; I feel this week could aptly be described as a lingering shadow of last week. For both better and worse, last week's "I started on" could basically be replaced with "I finished" or "I almost finished," and the log would be nearly complete. Thus, as has been falsely stated many times now, we may actually see a reasonably-sized devlog tonight!? (EDIT: at this point, it's a tradition...:ghost: )

---

PlateMeshes: 90+% complete in the LT core. As discussed last week, I emphasized building a release-quality implementation since they became an invaluable tool in LTC++. I now have a high-quality, fast, C library implementation of all of the old PM functionality with the exception of rotation of individual plates. I will need to spend a bit more time in order to finish that bit efficiently, but the rest of the work was by far the bulk. Frankly it required more time than I would have liked, but all of that time was spent adding supporting functionality to the core in order to efficiently support a good PM implementation. The good thing about this kind of time is that it's "permanent progress," as is all progress of the LT core (which is, as a reminder, designed to be the 'etched in stone' part of the LT engine).

Mesh 'Warps': 20% complete (I figured out how to do them). These are the things that allowed me to distort models in cool ways that added curvature and interesting global features to ships and stations. Unfortunately, their implementation in C++ using all that fancy (and incredibly-nasty) template stuff I used to do made it fairly easy. I had some trouble figuring out how to do the same thing effectively in a clean, C API (i.e., making it easy to build warping functions from LuaJIT). I've designed the implementation, about which I'm pleased. I haven't coded it yet. Interestingly, solving this design problem brought me insights on how to make reference-counting work cleanly across API boundaries (for example, to enable both the core and LuaJIT to hold pointers to the same object and still have all memory management work correctly). I've not had this insight before, but when the time invariably comes that I need to do this (beyond warps, which will require it from the get-go), I now know how to do it painlessly.

Networking & 'Control Panel': 100%/90% complete (server/client), and completely new to any of the LT engines! Yes, this is where about 2.5 days of my time went this week, but it's a technology investment that I'm betting will pay dividends. While working with planets, it occurred to me that I would really like a UI for changing the procedural algorithm parameters in real-time to see exactly how they're affecting the final result (with complex procedural algorithms it can be very difficult to reason about the effect of individual parameters). I didn't want to spend a huge amount of time building something just for this task, though, so, inspired by a friend at LTP who does web development, I decided to use a tool that already exists and is very fast and easy for creating quick UIs: the modern web browser. This necessitated networking functions in the LT core, which I had promised I would provide for modders who want to attempt 'multiplayer LT' mods anyway. So, I implemented a multi-platform TCP & UDP socket API, then used it from LuaJIT to spin up a very simple HTTP server run by in the background by the engine that serves up a simple control panel webpage when connected to via a browser. It can change internal engine variables automatically in response to controls like sliders or buttons being changed in the panel.

Though I'm hoping the ability to connect to and communicate with the engine via networking will pay substantial dividends over the remaining development time, having the control panel available is already really helpful: I have a really easy way to expose real-time-tweakable parameters, and, as a bonus, since it's based on sockets & HTTP, I can even connect remotely so that I can use one machine (like my tablet) exclusively as my 'controller' while I develop on the other. I'm also thinking this will be great for monitoring and managing long-running universe simulations (when I get to that point, ideally sooner rather than later)! I'd even like to explore using this for asynchronous resource generation on the GPU, which is traditionally difficult to do in a single process (due to threaded GL contexts being messy). There are a whole lot of ways I can envision using even this very basic networking tech :geek: Also, of course, the tools are now there (in theory) for anyone who wants to give an LT Multiplayer mod a shot at release (in practice, good luck, I recommend putting together a team :ghost:).

---

In the LTASM world, I did exactly what I had mentioned and hoped for last week: basic x87 floating point operations. Quite happy to have that stuff working. I've got enough to, for example, do most if not all of the basic FP math that you might do in the LT logic, although I don't necessarily have all of the opcodes for doing things in the most efficient way. Frankly, though, my first goal in this world of Plan Z is to have a script -> machine code pipeline working at all. That will be amazing enough for me, then we can worry about micro-optimization. My tests this week with FP instructions in simple loops are already blazingly-fast. I am getting a stronger and stronger feeling that this work will play a role in LT even if LuaJIT continues to hold. In fact, I can see it being a beautiful addition to LJ! LJ is basically a dynamic asm generator, why not have a static asm generator for perf-critical code? Man does it make a difference getting down to that raw level (I mean compared to something like Lua, not C/++).

Speaking of which, as usual, I spent some quality time with the Intel manuals this week, covering chapters 10 & 11 (SSE, SSE2), and skimming 12, 14, & 15, (SSE3/4, AVX/AVX2, AVX512). So basically, I learned a whole lot about vector / 'SIMD' instruction sets this week. Fun as always :geek: (Can you believe AVX512 has 2KB of register space for FP math?? Insane...)

---

It's worth mentioning that I spent a good bit of time this week on other, non-coding work. This week saw an abundance of emails, real-life events, and a bit more time spent with the forum than usual. Speaking of which, I assume that most of you have seen the announcement of Talvieno as Community Manager by now, but if not, check it out :) Although it took a bit of time this week to write that post and get everything squared away, his new role as CM will ensure that the amount of non-coding time spent during future weeks is kept to a minimum! I'm really happy to have Talvieno (officially) on board.

---

No, I'm not terribly impressed with this week (maybe I'm just tired right now...looking at the log, it seems fairly substantial?), next week should see less clerical work as well as less of having to finish up things, and more of getting to start new things. Seriously, next week I hope to see ships and stations in their full glory: PlateMeshes at 100% (rotations are definitely needed for stations), mesh warps implemented, procedural algorithms 100% ported, mesh ambient occlusion calculations 100% complete. I'd like to see asteroids as well, meaning scalar field extraction in the C core. If I have all of those things in a week, I'll be happy! I'll also have a good-looking, content-filled system, presumably. On the asm side, perhaps basic SSE functionality; let's say scalar operations at first, so that I can do the equivalent operations that the x87 FPU would do, but using the more-modern SSE (interestingly, the SSE instruction set is not just for vector processing, it has scalar (single-element) instructions as well).

Until some time in the next 7±2 days...! :wave:
“Whether you think you can, or you think you can't--you're right.” ~ Henry Ford
Post

Re: Tuesday, February 28, 2017

#7
Using the browser as remote control station is also what I have implemented.

Its just much easier to implement a remote-GUI in HTML instead of a native desktop GUI or have to use the command line, and you have the bonus to be able to administer the running application
remotely on a mobile device.
Post

Re: Tuesday, February 28, 2017

#8
Hey Josh, you mention that threaded contexts in OpenGL being "a bit messy". Have you considered Vulkan, since it's made from the ground up to be easily multithreaded? Plus I'm sure the performance boost would be nice as it's driver overhead is spread across multiple cores.

Unrelated, but will there be a Linux release of LT?
Post

Re: Tuesday, February 28, 2017

#9
Two-Tone- wrote:Hey Josh, you mention that threaded contexts in OpenGL being "a bit messy". Have you considered Vulkan, since it's made from the ground up to be easily multithreaded? Plus I'm sure the performance boost would be nice as it's driver overhead is spread across multiple cores.
That's been (seemingly) suggested here, but Josh hasn't addressed it specifically.
Unrelated, but will there be a Linux release of LT?
There will be a Linux release, yes. In fact, Josh is developing it on a Linux system. :)

Welcome to the forum, by the way. :wave:
Have a question? Send me a PM! || People talking in IRC over the past two hours: Image
Image
Image
Post

Re: Tuesday, February 28, 2017

#10
Forget multiplayer, I'm excited by the possibility of being able to leave LT running and remotely checking how my empire is running while I'm away, tweaking prices and ordering ships around via a web page while I'm at work. On lunch, obviously.

Years ago there was a mod that did some of this for (I think) X2 The Threat (definitely one of the games in the series). I found it pretty cool that I could do stuff remotely on my universe sim.
Post

Re: Tuesday, February 28, 2017

#11
Talvieno wrote:That's been suggested here, but Josh hasn't addressed it specifically.
I hope he does! Vulkan has a lot of benefits he could use; lower frame latency, lower cpu usage, less time spent on the cpu during calls (IIRC, several times lower), VERY multithreadable, handles high number of objects far better, and more.
There will be a Linux release, yes. In fact, Josh is developing it on a Linux system. :)
frakking awesome.
Post

Re: Tuesday, February 28, 2017

#12
Triscopic wrote:Forget multiplayer, I'm excited by the possibility of being able to leave LT running and remotely checking how my empire is running while I'm away, tweaking prices and ordering ships around via a web page while I'm at work. On lunch, obviously.

Years ago there was a mod that did some of this for (I think) X2 The Threat (definitely one of the games in the series). I found it pretty cool that I could do stuff remotely on my universe sim.
This!

Who cares about multiplayer when I can check in via my phone or the game can push messages to me if they are important so I can make a choice while not being at the game :D
procedurally generated comment
Post

Re: Tuesday, February 28, 2017

#13
JoshParnell wrote:Tuesday, February 28, 2017
As discussed last week, I emphasized building a release-quality implementation since they became an invaluable tool in LTC++. I now have a high-quality, fast, C library implementation (...)
It may be a dumb question, but did you ever consider the language "D", Josh?
Automation engineer, lateral thinker, soldier, addicted to music, books and gaming.
Nothing to see here
Flatfingers wrote: 23.01.2017: "Show me the smoldering corpse of Perfectionist Josh"
Post

Re: Tuesday, February 28, 2017

#14
JanB1 wrote:
JoshParnell wrote:Tuesday, February 28, 2017
As discussed last week, I emphasized building a release-quality implementation since they became an invaluable tool in LTC++. I now have a high-quality, fast, C library implementation (...)
It may be a dumb question, but did you ever consider the language "D", Josh?
How about we keep all languages out of his knowledge, to prevent further delays. :D :ghost:
Resident derp.
For some reason, I feel obliged to display how many people have talked in IRC over the past 2 hours: Image
Image

Online Now

Users browsing this forum: Google [Bot] and 1 guest

cron