Engine Dev

There's only need for one forum for now. Discuss development. No advertising or thinly veiled attempts at advertising. Create a single thread for your project and be prepared to show and tell.
Post Reply
User avatar
ShortroundDev
Posts: 9
Joined: Mon Apr 05, 2021 10:14 am
Location: Virginia, USA

Engine Dev

Post by ShortroundDev »

Hi, I've been developing my own 2D engine for a year or so now. I'd like to tell you about it and see other engines people have made

CPhys is a 2D game engine written in C# (.Net Core 5) using a forked version of SDL for Rendering, Audio, and I/O. I say forked because I forked it and added some new functions to it.

Entities in my engine are C# classes implementing a hybrid ECS and inheritance system. Entity components are their own classes and can be assigned to any entity implementing its associated interface. For example, while a Zombie may override the "Creature" class, which defines some basic rendering and physics checking, assigning "Burnable" to it, and giving its Mixin Dictionary the "BurnableMixin" object adds the code necessary to make the Zombie ignite when hit with a fireball.

AI for the entities is a Finite State Machine; each state is defined as a method in the Entity's class, and is adorned with an Attribute describing it as both an AI state (called a "Goal") and also some meta information about it (such as what "Target" entity should be passed to the method, making it simpler for enemy Entities to target the player). Goal methods can return things like

Code: Select all

Next("Walk")
to transition to the "Walk" state or

Code: Select all

Frame()
to loop through the same AI state again.

Collision detection in my engine is a kind of convoluted Swept AABB system. I don't totally love this system but it works well enough and changing it would probably be a headache at this point. There are no tiles in my game, everything is an AABB entity of arbitrary dimensions and position. This allows me to do things like moving platforms pretty simply. Entities are placed in 1024x1024px 2D buckets for rendering, and their position in a Quadtree is calculated on the fly for collision detection.

Rendering is done with SDL2. Having learned more about SDL2 as I went through this engine, if I could do it again, I would implement the rendering in DirectX 11 instead. I feel like SDL does some things a little too slow (such as tile-based rendering). Though my spatial partitioning isn't tile-based, I do have wall entities whose sprites which are defined by tiles which are atlased from a spritesheet. Because SDL doesn't currently have a way to specify multiple UV-mapped quads in a single draw-call, I implemented this myself, which was a pretty good performance boost. This AABB entity system makes it simple to move geometry around at-will, but also makes it cumbersome to define arbitrarily shaped geometry. I'm working on an autotiling system to generate static AABBs to make the workflow a bit easier.

UI is currently an XML-based declarative syntax, similar to XAML or HTML. Models can be bound to these using Handlebars. Stylesheets for the menus are defined in JSON, similar to CSS. Callbacks for each element are a C# object defining methods like "OnClick", and "OnKeydown". This system was designed ad-hoc and isn't very strongly typed, so I'm refactoring it to have better typed definitions of what styles are available, as well as a more flexible callback system similar to Angular. Stylesheets were transitioned from JSON to YAML because I like the syntax's lack of quotation marks better; it's closer to CSS and more flexible that way, plus it allows better syntax highlighting.

Dialog is currently basically just stored XML files which define the UI for the various dialog boxes. Dialog boxes and menus are essentially the same thing, and I have a webapp which generates those menu files from dialog I type in and branching trees. I dislike this system a bit because the workflow is slow, requiring me to export from one program and import into another. The new UI system will enable me to edit dialog trees in-engine so that I don't have to deal with this export/import stuff. Since the game will focus heavily on dialog (not quite story-driven, but more of a logic puzzle adventure-type almost point-and-click game), it's important that the workflow for this work well.

Scripting is done in various places (such as in dialog/cutscene events, or in response to buttons being pressed), via Lua. I added a whole suite of Lua functions to enable scripts to control entities, force them into certain AI state, move them in certain directions, etc. I'd like to flesh this system out a bit better as well. I also have a Lua Interop library for debugging, but because this gives direct access to all of the .Net library, this can't be used for scripting due to security reasons, and I may have to move it to a non-interop version later on due to its ability to be abused via config files executing arbitrary code.

There's also cutscenes, which are defined as a series of keyframe transformations (camera pan, zoom, entities being manipulated, dialog being triggered, and lua scripts being executed), which is defined in XML.

There's quite a lot more, but these are the core features. Here's a screenshot I took of the editor, with all the editor menus open:

Image

The dots are a resizable 32x32px grid, and the blue lines denote the edges of each bucket. The UI rendering is all done in SDL via my custom UI framework
:geek: :ugeek:
User avatar
Deckhead
Site Admin
Posts: 128
Joined: Fri Feb 21, 2020 5:44 am
Location: Sydney, Australia
Contact:

Re: Engine Dev

Post by Deckhead »

Firstly, well done. I know first hand just how much effort you've put into this, and I know how great it feels to have something working well.

Secondly, I don't mean this to take away from your achievement, but I have to ask, are you planning on making a game, or an engine? It's a hard one, and I know how easy it is to keep working on the engine. I will say though, that the engine is the best part, and there's nothing wrong with never making a game (like me lol).

I have a some questions/comments:
  • Why did you go with the mixed inheritance / ECS method? I've done both, but never thought to combine them. Do you have the capability to iterate over entities based on their components, if so, are they sharing data locality?
  • For your rendering, are you making use of the ECS to perform it? I.e., iterate over all "renderable" components and draw them? Because if you are that should allow you to kind of batch the rendering into a single draw call fairly fast. I know SDL might not be capable of it but you said you were forking it.
  • Actually, why fork SDL? You can write your own renderer pretty easily. You've got the technical chops to handle it, and it looks like you really just need a bucketing system, maybe with an z-depth sort prior to a single draw call. It's what I do in my engine, all draw calls are bucketed and then sorted before being dispatched in as few calls as possible (bucketing based on texture and shader, ordered by depth). It's blazingly fast for me.
  • Can you tell me more about your UI? I coded my own UI system using a widget model. I also coded my own scripting engine that can interact with it. But it looks like you're using various web techs there. Are you using external libraries for that?
Developer of The Last Boundary and webmaster of IndieGameDev.net
User avatar
ShortroundDev
Posts: 9
Joined: Mon Apr 05, 2021 10:14 am
Location: Virginia, USA

Re: Engine Dev

Post by ShortroundDev »

Deckhead wrote: Mon Apr 05, 2021 11:28 pm
Secondly, I don't mean this to take away from your achievement, but I have to ask, are you planning on making a game, or an engine? It's a hard one, and I know how easy it is to keep working on the engine. I will say though, that the engine is the best part, and there's nothing wrong with never making a game (like me lol).
Both. I like reinventing the wheel, and don't feel like making a game without making my own engine. I've played with other engines before but nothing feels as fun to develop as my own
Why did you go with the mixed inheritance / ECS method? I've done both, but never thought to combine them. Do you have the capability to iterate over entities based on their components, if so, are they sharing data locality?
It started out with an inheritance system, as my previous engines have. I worked for a web dev company that did something similar to ECS using reflection in Java, implementing a kind of extension method system. Initially, my components were just extension methods on interfaces (since default-implementation methods didn't exist on interfaces in C# at the time). This got messy and ugly. As I learned more about engine development, I learned about ECS and decided to use it for the more reusable stuff (which entities are burnable, killable, etc.), and to keep the logic specific to the particular enemy (zombie, skeleton, etc.) as part of the inheritance system, rather than having "singleton" components which are only ever assigned to a single type of entity.
For your rendering, are you making use of the ECS to perform it?
No, all entity's have a Draw() method, but not all of them implement it. Since this is a 2D engine, I have a lot of wiggle room for performance, so I simply iterate all visible entities and call their draw method. There's a sprite/animation rendering system that all entities have access to (and almost all of them use, except very special entities), and I just created a child class of entity called "Creature" which uses that drawing by default. Creatures are entities which actually show up in the level (Zombies, the player, Boxes, Switches, etc.), and things which inherit the Entity class directly are usually special kinds of entities, such as teleport locations and such.
Actually, why fork SDL? You can write your own renderer pretty easily. You've got the technical chops to handle it, and it looks like you really just need a bucketing system, maybe with an z-depth sort prior to a single draw call. It's what I do in my engine, all draw calls are bucketed and then sorted before being dispatched in as few calls as possible (bucketing based on texture and shader, ordered by depth). It's blazingly fast for me.
1. When I first started this project, I didn't know OpenGL or DirectX. I had only ever used SDL. As I went on, I pushed up against the limits of what SDL supports (AABBs only), and by that point most of the rendering code was dependent on SDL types. (SDL_Texture, SDL_FRect, etc.), and I didn't feel like refactoring it.
2. SDL does 99% of what I need it to do (simple AABB rendering), and it does it fast enough for me. If I were to start over from scratch, I'd probably do it all in DirectX, but I'm not restarting at this point. I know enough about how SDL works on the backend that when I need it to do something (like drawing arbitrary quads/triangles, or tile-based rendering), I can just open up SDL and implement it myself, then add the bindings to the C# bindings library I use. Maybe someday I'll push these back up into the SDL repo (which is now maintained via github), but I'm not supporting Metal or Vulkan, so someone else would have to finish those up for me. I don't LOVE my rendering system, though I am proud of having forked SDL for the things I need it to do. It does what it needs to do (though the default orthogonal projection means that parallax backgrounds need to be implemented in software, dividing x and y coordinates by the depth). I may refactor my camera system to take advantage of their new batched rendering so that I can just transform the projection matrix by a camera transformation matrix once before rendering, rather than having to add Camera.X and Camera.Y and multiplying Camera.Scale for every single draw call, which is a PITA.

(I'd also probably replace my SDL_Point calls and custom Point geometry struct with some native SIMD vector in C#
Can you tell me more about your UI? I coded my own UI system using a widget model. I also coded my own scripting engine that can interact with it. But it looks like you're using various web techs there. Are you using external libraries for that?
Sure. my current UI system (that I'm refactoring and phasing out) uses only 1 external library: Handlebars.Net. Handlebars is a web technology meant to bind models to templates. This means that if you've got some object like:

Code: Select all

{
    "Name": "Bill",
    "Age": 26
}
and a UI template (HTML, XML, XAML, etc.) like:

Code: Select all

<MyMenu>
    <Name>{{Name}}</Name>
    <Age>{{Age}}</Age>
 </MyMenu>
 
Binding the model to the template will create text with the values filled in from {{Name}} and {{Age}}. Handlebars supports some basic loops and conditionals (and can also evaluate expressions and sub-expressions in a lisp-like polish notation), so if I need to show my inventory, I can have a template like:

Code: Select all

<Inventory>
    {{#each Inventory}}
    	<InventoryItem>
    	    <InventoryImage Src="{{this.Image}}"/>
    	    <ItemName>{{this.Name}}</ItemName>
    	</InventoryItem
    {{/each}}
</Inventory>
And when I want to show the player's inventory, I can just do something akin to:

Code: Select all

ShowMenu("/Path/To/Inventory/Template.hbs", Player.Inventory)
.

I style these menus in a Json file which looks like a verbose CSS file. I can query for the UI elements using their Tag name, their ID Attribute, or their Class Attribute. Each entry in the Json file is a dictionary of style, so if I want to set the inventory menu's size to 800px by 600px, it would be:

Code: Select all

{
	"Inventory": {
		"Width": "800",
		"Height": "600"
	}
}
If I wanted it to be 100% of the screen width, I'd replace "800" with "100%", for example. There's all kinds of styles, like border size/color, font, font color, background color, background image, Z-index (to determine draw order), all kinds of stuff taken from CSS.

In order to make these menus DO something, I've got callback objects. I define a class which overrides the "ControlCallback" class. This class defines an interface of methods like "OnClick", "OnMouseIn", "OnMouseOut", "OnKeyDown", etc. The overridden method then determines the logic for the corresponding event. If there's an OnClick callback object for some XML Element, then it will run that OnClick() method.

These callbacks are associated with the XML Element via a C# attribute [ControlTarget(QUERY_STRING_HERE)], so if I wanted to add an OnClick callback for the InventoryItem, it might look like:

Code: Select all

[ControlTarget("InventoryItem")]
public class InventoryItemCallback : ControlCallback
{
	// ClickedElement is the XML element which has been clicked on (called a "control")
	// SDL_Event is the raw event Info
	public override bool OnClick(Control ClickedElement, SDL_Event E)
	{
		Console.Out.WriteLine("Foobar");
		return true; // return value determines if events bubble up
	}
}
I'm refactoring and hopefully eventually scrapping this current system. The new system is in the works and I've already got it mostly fleshed out, I just need to code it. This will also be a separate DLL so the hope is I can then use this system to create tools for my game in the same environment/engine. Currently, my tools are all done in HTML/Typescript.

While I work on the engine, the game isn't getting made. That's OK. I use this time to write down ideas, and throw out old, bad ones. I also sometimes go work on animation and drawing stuff, it's not ALL coding.
:geek: :ugeek:
User avatar
Deckhead
Site Admin
Posts: 128
Joined: Fri Feb 21, 2020 5:44 am
Location: Sydney, Australia
Contact:

Re: Engine Dev

Post by Deckhead »

ShortroundDev wrote: Tue Apr 06, 2021 12:44 am and to keep the logic specific to the particular enemy (zombie, skeleton, etc.) as part of the inheritance system, rather than having "singleton" components which are only ever assigned to a single type of entity.
If you commit 100% to an entity component system, this generally won't occur. It's hard to describe, but if everything is components of entities, you generally never have a component that's only used by one entity ever. Even things like "zombie" or "skeleton" are no longer components, they are broken down to "killable", "ai", "renderable" etc.

For me, it took a long time to fully appreciate how to use component based systems for a game. I really had to rethink a lot of assumptions about how things work.

It sounds like you've made the right call with SDL. If you've spent that long mastering it, you may as well continue with it.

For the UI system you've got in place, that's pretty much what I've done. I rolled my own though, which wasn't the best idea, but in the end it works well and gives me the output that I want.

The UI elements themselves though, have you coded the necessary things like "button", "window" etc or are you able to reskin an existing system? I ask, because your screenshot has a very complete UI system, and I'm sort of 2/3rds towards that (having coded my own).
Developer of The Last Boundary and webmaster of IndieGameDev.net
User avatar
ShortroundDev
Posts: 9
Joined: Mon Apr 05, 2021 10:14 am
Location: Virginia, USA

Re: Engine Dev

Post by ShortroundDev »

Deckhead wrote: Tue Apr 06, 2021 1:06 am If you commit 100% to an entity component system, this generally won't occur. It's hard to describe, but if everything is components of entities, you generally never have a component that's only used by one entity ever. Even things like "zombie" or "skeleton" are no longer components, they are broken down to "killable", "ai", "renderable" etc.
I trust that you're right, but that would be too cumbersome to refactor at this point, and the current system works pretty well for what I need it to do. In a future engine, I'll probably go full ECS.

The UI elements themselves though, have you coded the necessary things like "button", "window" etc or are you able to reskin an existing system? I ask, because your screenshot has a very complete UI system, and I'm sort of 2/3rds towards that (having coded my own).
Buttons and other core UI elements are implemented in the UI system, not the engine. By this, I mean that an element like <button>Click me</button> is indistinguishable from an element like "<PlayerHealth>70 / 100</PlayerHealth>". They're both XML elements with various styles and callbacks associated with them. When I need a button to do something specific, I give it a callback defining its specific behavior. If I need the button to share some general color or background or border with another element (such as the classic Steam green I use for the level editor), I assign those styles to a class, such as <button Class="UI"> or <Panel class="UI">. Every Button has a callback which defines the stylistic behavior (button clicking effect) when clicked (it swaps the top-left and bottom-right borders).

I come from the web dev world, primarily, so the system is designed to be pretty close to a web browser, but with things I like about XAML as well. I'd ultimately like to add a custom scripting environment for it (probably either Lua or C#), but that would require a herculean effort to add an entire useable API. This would enable 3rd parties to add content to the game without having to recompile the engine.
:geek: :ugeek:
User avatar
Deckhead
Site Admin
Posts: 128
Joined: Fri Feb 21, 2020 5:44 am
Location: Sydney, Australia
Contact:

Re: Engine Dev

Post by Deckhead »

ShortroundDev wrote: Tue Apr 06, 2021 1:31 am I come from the web dev world, primarily, so the system is designed to be pretty close to a web browser, but with things I like about XAML as well. I'd ultimately like to add a custom scripting environment for it (probably either Lua or C#), but that would require a herculean effort to add an entire useable API. This would enable 3rd parties to add content to the game without having to recompile the engine.
Along these lines, I toyed with the idea of using chromes renderer in a game engine. Apparently it's an isolated system that will take HTML (and potentially javascript?) and make the necessary draw calls. You just need to provide the draw calls. This was for a very UI driven game, based around internet browsing, so it made sense. In the end, I dropped the game, but there's that option.
Developer of The Last Boundary and webmaster of IndieGameDev.net
User avatar
ShortroundDev
Posts: 9
Joined: Mon Apr 05, 2021 10:14 am
Location: Virginia, USA

Re: Engine Dev

Post by ShortroundDev »

Deckhead wrote: Tue Apr 06, 2021 1:49 am Along these lines, I toyed with the idea of using chromes renderer in a game engine. Apparently it's an isolated system that will take HTML (and potentially javascript?) and make the necessary draw calls. You just need to provide the draw calls. This was for a very UI driven game, based around internet browsing, so it made sense. In the end, I dropped the game, but there's that option.
Yeah, you can create any webapp as a standalone program using electron, which lets you basically open up a headless version of chrome. Discord, Slack, lots of applications are just web applications, even VS Code. You can also design very performant graphical applications as well; HTML5 Canvas + Javascript enables an SDL-like library of graphical functions (akin to a fixed function pipeline), and there's also webgl, which allows performant 3D applications with GLSL Shaders. Three.js is a popular high level javascript library for WebGL. Chrome (and all other browsers) also have Websockets, which would allow fast multiplayer games, as well as WebRTC, which I THINK can be used to do peer-to-peer games as well. There's also a gamepad API which you can use for controller support for games over bluetooth or usb.
:geek: :ugeek:
User avatar
Deckhead
Site Admin
Posts: 128
Joined: Fri Feb 21, 2020 5:44 am
Location: Sydney, Australia
Contact:

Re: Engine Dev

Post by Deckhead »

ShortroundDev wrote: Tue Apr 06, 2021 2:13 am
Deckhead wrote: Tue Apr 06, 2021 1:49 am Along these lines, I toyed with the idea of using chromes renderer in a game engine. Apparently it's an isolated system that will take HTML (and potentially javascript?) and make the necessary draw calls. You just need to provide the draw calls. This was for a very UI driven game, based around internet browsing, so it made sense. In the end, I dropped the game, but there's that option.
Yeah, you can create any webapp as a standalone program using electron, which lets you basically open up a headless version of chrome. Discord, Slack, lots of applications are just web applications, even VS Code. You can also design very performant graphical applications as well; HTML5 Canvas + Javascript enables an SDL-like library of graphical functions (akin to a fixed function pipeline), and there's also webgl, which allows performant 3D applications with GLSL Shaders. Three.js is a popular high level javascript library for WebGL. Chrome (and all other browsers) also have Websockets, which would allow fast multiplayer games, as well as WebRTC, which I THINK can be used to do peer-to-peer games as well. There's also a gamepad API which you can use for controller support for games over bluetooth or usb.
I just did a quick google search. I was thinking of Chromium Embedded Framework, which I believe is somewhat in the vain of Electron. Apparently this is what Unreal Engine uses.

Anyway, keep posting updates about your engine. I love to hear about this sort of stuff. I'm feeling inspired to make a post with my own engine.
Developer of The Last Boundary and webmaster of IndieGameDev.net
User avatar
ShortroundDev
Posts: 9
Joined: Mon Apr 05, 2021 10:14 am
Location: Virginia, USA

Re: Engine Dev

Post by ShortroundDev »

Image

Update, I can now define button UI as such:

XML:

Code: Select all

<button Class="Ui" />
YAML (stylesheet)

Code: Select all

.Ui:
    BackgroundColor: $4c5844ff
    BorderSize: 1
    BorderColor: $889180ff, $282e22ff, $282e22ff, $889180ff
    Color: $d8ded3ff
    
button.Ui:
    MouseDown:
        BorderColor: $282e22ff, $889180ff, $889180ff, $282e22ff
by adding a "MouseDown" stylesheet to the "button.Ui" stylesheet (which is a query string; that's supported now too), I can define the behavior for what happens when the player clicks on something; I can also use the "Hover" stylesheet to say what happens when the mouse hovers over.

Previously I had to write a callback in C# for every UI Element that had some kind of style change when clicked. Now it's all declarative
:geek: :ugeek:
Post Reply