I just finished reading posts of this thread and here are my thoughts and ideas.
But before that I'd like to tell a bit about myself and my experience. I have about 10+ years of C/C++ and 5 years of C# coding. Also have quite some experience in other languages like Python, Java, etc. Now I'm using Rust as my hobby language. I was working in a different types of companies, mostly server side backend projects in C/C++/C#. Even in one game company at the beginning of my career
. In my current company I'm working on AR project on Unity.
I don't have much time but depending on situation can spend several ours per week for this project.
First what I'd like to say is to make this project run properly organizational structure is most important. What I mean by this is that there should be a core team that will make a final decisions about what and how to do. This should be a team of designers, architectors and developers that will define a direction in which project should go. Of course it's should not be dictatorship and they should listen to others but final decision should be done by a dedicated team.
Otherwise I had a lot of examples when even in a big and solid company people can f**k up a project if there is no responsible manager/architect to define a proper roadmap for a teams. In such a situation different people and teams do whatever they wants and at the end different parts of the project just don't work together.
This core team should not be very big and people of course will join and leave it but it should be.
Now I'd like to talk a bit about technical things. Probably at this moment it's more reasonable to talk about higher level design and gameplay but I think that some technical moments can (and should) be discussed now as well.
First thing is a core of the game engine. As it was already mentioned by others for the game of such a size performance is very critical. I think nobody wants to repeat Josh's path. So as a core language we need something that gives the highest performance. This requirement immediately throws away most of garbage collected and scripting languages. So for me only C/C++/Rust and similar languages will be suitable. I would prefer Rust despite my big experience with C/C++ and about my reasons I will talk below.
Second thing is modularity/moddability. As other people already said in this thread this is a collaborative effort and we need this project to be modular. To achieve this we have to choose what communication interface to use between this modules. Only option I see in sense of speed and support by different languages is C FFI.
Also I think we have to support 2 types of modules: scriptable and native. Scriptable modules will be used for quick prototyping and and not performance critical things. So one can write a module in Python or Lua and quickly test it and if it works properly rewrite it in low level language if needed. The other idea that I have is that we can embed some modules in the game core. The way like it'd done in Linux kernel: you can load module dynamically but also you can embed it in the Linux kernel to gain better performance by avoiding intermediate layer. But this is just my idea.
As third, I think it should be decided at early stage whether to use existing game engine or to write custom one. Both of them have their pros and cons.
Game engine pros:
- A lot of architectural design decisions already made and a lot of code written.
- Most of existing engines have a good tooling and IDE's.
- Big engine community support.
- More developers that know this game engines are out there that can join the project.
- It would be hard to optimize existing engine to the desired scale of our project.
- Programming language usually strictly specified by language. Some engines allow to use other programming languages but usually with performance and/or usability cost.
- We will have to deal with a bugs in those engines. Some of them have a big delay in bug fixing, some are not open sourced and it would be hard to track those bug down.
As an option Unreal, Godot or similar engines with a low level core languages can be selected. I would not advise to use Unity despite it's IL2CPP backend. What I see in my currnt project IL2CPP of course speeds up a code but steel not enough. It's ok for small and some middle size of data but when it comes to customer data size we still have a performance problem. But it's only my experience.
Custom engine pros:
- We can make it as fast as possible.
- We will choose in which language to develop it.
- Architecture of the engine can be developed with taking in account all desired features.
- A lot of architecture design and coding work. Can be hard to support a big codebase.
- Can became a mess at the later stages of development if wrong architectural design decisions will be made at the beginning.
At the end I'd like to explain why I think Rust is a better option as a core programming language for a custom game engine.
As I mentioned above I wrote a lot of C/C++ code but what learned from it is whatever coding practices you follow, how hard you can try to make code correct you will always have all those annoying crashes, memory issues and my favorite (and hardest) multithreading bugs. And when project grows it became worth: different people have different styles of coding and not everybody follow advised coding practices. And simply you can be tired and miss or forget something. We are all human at the end.
I suppose some of you had such an experience when in a big company you have big and old codebase and main principle is: it works? - don't touch it. And if you have to change some code you usually copy paste some function instead of refactoring it. Bug fixing can be even more "exciting" (I remember one multithreading bug my team was hunting for a several weeks).
And here we come to Rust. I don't want to say it's a silver bullet but it let developers avoid a big amount of common problems: most of memory issues and also data racing problem. So here are some pros and cons of Rust from my point of view.
- Prevents most of memory issues and data races problem at compile time. Developer just won't be able to compile code if it contains such an issues.
- As an consequence of the previous pro it is much easier to make modification in the existing code: you can be sure that your changes won't introduce any memory or data race problems (of course you can still introduce logical bugs but that's why we write tests
- Rust has a very good type system and because of that its compiler can generate very optimized code. Speed of the generated code if comparable with an equivalent C code or sometimes even faster.
- Very good and easy to use package management system (similar to npm).
- Relatively hard language to start with. Learning curve is pretty steep for the most of developers.
- Not so much developers knowing Rust at the moment.
Don't take this as advertisement of Rust. It's just my thoughts.
It was pretty big post but I hope it will be helpful for someone.