Géza's Stuff
>_

Magma3D Dev-Log 1

2024-07-12

Last year, I started developing my own game engine using Rust. At this point, I had already done some game dev in Unity and had thought about switching to GODOT. But because of the type of game I wanted to develop (a SPACE game), I had to constantly fight Unity's default physics implementation.
This is how the idea of developing my own engine was born.

Rust...

If I said Rust was the best language for everything, I would be lying. But personally, I use it a lot, mainly because of the compiler (it even warns you about the Greek question mark).
Because (almost) everything has to be specified by you, it forces you to really think about the code you are writing. In my experience, this has led me to catch a lot of bugs even before compiling, which removes a lot of the pain you get using languages like Javascript or Python.
But this isn't always important! Sometimes it's worth it to trade this safety for speed and simplicity.
Anyways, I'm using Rust and I haven't regretted that decision until now. So let's dive into the actual topic!

The Perfect Game-Engine

It all started with not one, but a couple of ideas. I had been using Unity for game development for some time, but my experience had not been great. After desperately trying to bend the default physics implementation to my needs, I realized it. It wasn't that the engine was bad; it just wasn't made for the sort of thing I was trying to do.
So an idea started to form in my head.
I was going to make my own engine, perfectly suited for my use case. In my enthusiasm, I made a plan and wrote down some requirements.
This was going to be my magnum Opus!

Requirements:

- High-detail voxel rendering
- Advanced physics simulation
- Networking
- VR-support
- Using other languages than Rust with WebAssembly
- Also has to run on potatoes

I was also fascinated by the ECS architecture, especially after seeing what Bevy did with it. So, like Bevy, I was going to have an ECS as the basis of my engine.

To learn more about why this is cool, read this.

Building an Entity-Component-System (Was Somehow Easy)

After learning about ECS, I had to learn how to implement it.
My plan (which maybe wasn't the best) was to first build a simple ECS to learn how it works, and then expand it mainly with multithreading. The first part of the plan went quite well. I also found a lot of resources on the topic, which made things easier.
The first site that popped up after a quick search was this blog post by Ian Kettlewell, which provided me with a good grasp of the basics.
So I spent a day or two reading and following tutorials until I had a working prototype.
Well... This went a bit too smoothly, I thought. Something had to go wrong. But somehow it actually didn't.
...for now

You can find my ECS in its current form here.

What Bevy got right

Earlier, I briefly mentioned Bevy, but what I didn't mention was its influence on this project.
For those uninitiated, Bevy is another (much better than mine) game engine written in Rust that also uses an ECS as its base architecture. I first heard about Bevy a while before starting this project, which was the reason why I was excited for ECS in the first place. When planning out the architecture of my engine, Bevy inevitably came up again. I knew it was good, but until then I hadn't realized how good it actually was. The way their ECS works and is integrated with every functionality of the engine blew my mind.

After I had my prototype ECS, I started implementing engine functionality on top of it, orienting myself a lot on how Bevy did things because I just couldn't think of a better way. The most important part of Bevy I copied is their plugin system (although I called it modules).
I told you that I liked how everything in Bevy works with its ECS. The way they make that possible is through the plugin system. Basically, "Plugin" is just a trait that requires you to implement a "build" method. In the build method, you tell Bevy what it has to do to set up the plugin. When you now add the plugin to Bevy's World (where all the application's data resides), it just executes the build method, and the plugin setup is complete. Simple, right?
That is essentially what I did too. First, I implemented the App struct, which is basically a wrapper around the ECS World. It provides a way to add Modules (my plugins) to your app and also set a custom runner.


// lib.rs

pub struct App {
    pub world: World,
    runner: fn(App),
    modules: Vec<TypeId>,
    startup_systems: Systems,
    update_systems: Systems,
}

impl App {
    pub fn add_module(&mut self, module: impl Module + 'static) {
        let type_id = module.type_id();
        if !self.modules.contains(&type_id) {
            self.modules.push(type_id);
            module.setup(self);
        }
    }

    pub fn set_runner(&mut self, runner: fn(App)) {
        self.runner = runner;
    }

    // ...
}
                

The code for the modules looks like this:


// module.rs

pub trait Module {
    fn setup(&self, app: &mut App);
}
                

In my case, each Module must provide a setup method, which takes a mutable reference to an App.
As you can see, it is indeed quite simple. But very powerful.


// Defining a Module and adding it to the App

let mut app = App::new();
app.add_module(ExampleModule);

struct ExampleModule;

impl Module for ExampleModule {
    fn setup(&self, app: &mut App) {
        // Setup the module
        // E.g. register components to the World or add resources
    }
}
                

This can now be used to integrate engine functionality with the ECS, but it also allows the user to define their own.
If you want to know how Bevy's Plugins work, have a look at this. You will see that it is nearly the same (from an API perspective).

Winit

Now that all the groundwork was laid, I could finally start developing the actual engine. Among a small audio module integrating kira (just bare bones for now) I also had to have some way to create and manage windows. winit is the rust library for that, so I just had to write a simple integration...

...Winit itself is a nice library if you are writing an application. That is, it provides you with an event loop, and you just have to match the events important to you. But problems arise when one wants to integrate the winit events into an ECS (for me at least). I struggled quite a bit with making the events available to users of my engine. In the end, I settled for a suboptimal solution of using winit's pump_events. I hope to improve this in the future, and while writing this, I just had an idea of how. Instead of making the events available to the user, I will let the user write handlers for different events, more similar to matching.

UI

I tried writing an iced integration, but that was very rushed and is so bare bones you might as well just use iced standalone. In the future, I'd like to write a proper UI integration (or even my own toolkit). For now, though, this will have to suffice.

Remember when I said building an ECS was easy?

So... I had just started development on the renderer (I'm using wgpu), when I realized something: Maybe I should at least somewhat finalize the ECS first so that I don't have to rewrite everything in the end because of API changes to the ECS. I stopped everything I was doing and returned to the ECS.
The main thing still left to do here was multithreading. I knew it was going to be difficult, but I hadn't anticipated the scope of it. To make things worse, I also had never used Rust's concurrency features before, and after trying and failing miserably multiple times, I kind of just gave up.
Until I had a random shower thought half a year later, that seemed to be the solution. When first trying, I had read this blog post, and so I tried to implement the same approach. Wrapping my entities in an RwLock to make them thread save. Obviously, I had failed the first time around, but now I knew how to do it right. In the end, it took me just a couple of hours to do it. This was accompanied by some API changes, so I am currently adapting the engine to these changes.

I will write more about how I actually achieved multithreading in my next post. And maybe I'll then be able to show you some progress on the renderer, which I have some ambitious plans for.