Entitas Modules / Packs / Kits

[sschmid]

I think that is actually the next big cool feature if I manage to find a solution. I was thinking about Entitas and modules / packs / Kits and how one could tackle this.

The goal

Someone writes a bunch of systems and components, packs them and calls it PathFindingKit or PhysicsKit or AnimationKit. Basically a self contained and tested module that you drop into your game and voila you have PathFinding, Physics or Animation.

---

Kits most likely come with their own contexts and components, so first of all I have to find a solution for #303 (Support for multiple contexts in ReactiveSystem).

---

Vision

You download a PathFindingKit. I comes with components, systems and contexts. You install / register this kit somehow with Entitas and declare that your own GameEntity is also a PathFindingEntity (defined in the Kit). Now your GameEntity supports methods like e.AddPath() and things like that.

---

It's an interesting though which would enable the Entitas community to share Kits which solve certain problems. We basically could create small reusable building blocks and put them together in our games

[AGeorgy]

Awesome idea!!
It reminded me of:

1. https://github.com/PlayMakerEcosystem
2. https://hutonggames.fogbugz.com/?W1181

[AGeorgy]

It #309 It can help in the dependencies of another Kits

[mzaks]

I gave the whole idea some thoughts which I want to share.

As I see it there are two ways how a Kit can be implemented.

1. Actor Model approach. (https://en.wikipedia.org/wiki/Actor_model)
   A Kit is a self sustained package. It has it's own set of contexts with resulting Matchers and Entity types. It has it's own set of components and systems. As a user we have to instantiate it's context and the root system of the kit and trigger it in our game loop. We interface with the Kit by using the context to create it's entities and set components in it.
2. Context Extension/Inheritance approach.
   A Kit provides base functionality. It defines Components and Systems which we can use by extending our context with it.

Both approaches has complications.
In Actor Model approach we would have to bridge data from our context to the Kits context. Which can lead to tedious mapping code.
In Context Extension approach we will basically end up with multiple inheritance and lots of inefficiencies. Because if Kit A contains 20 components and I extend my context with Kit A I am inheriting all 20 components and my Entity get's 20 more slot's in the underlying array. Even if I use only 3 out of them. As always Pareto principal (https://en.wikipedia.org/wiki/Pareto_principle) applies. Only 20% of the components will be needed by 80% of the users meaning that everyone will put a 80% overhead in there projects by Context Extension approach.

I personally would rather try to tackle the problem of entity mapping. We could introduce a mechanism which a Kit developer could introduce in order to make the mapping automatic. Like for Example provide an interface which my components have to implement in order for the kit to run through an "unknown" entity and extract data which it needs.

So for example Kit has a PositionComponent and it implements a Kit.IPosition interface. If you have your own PositionComponent just let it implement this interface and than you can pass your entity to Kit's context and Kit's context will be able to create it's own entity based on yours (or sync it's entity with provided). So a sync method on entities which can receive a generic entity.

I believe actor model is much more superior to the inheritance model in simplicity, but than again I am biased :)
https://medium.com/@icex33/consider-oo-inheritance-harmful-4612e65f3047#.engi9dm7h

[sschmid]

First proof of concept!

[sschmid]

TL;DR
Currently struggling. I'll probably have to step back and check if there are other solution or try again on the weekend.
If you guys have any suggestions or ideas how to solve this, let me know!

The setup:

I've create a new C# solution with 2 projects

1. PhysicsKit (name ist just a placeholder)
   This project contains 2 components in the namespace PhysicsKit

• PositionComponent
• VelocityComponent

2. MyGame project
   This console application project contains 1 components and serves as a test project to integrate the PhysicsKit.

I updated the project's Entitas.properties with new Kit syntax. When specifying contexts you can optionally register Kits like this

Entitas.CodeGeneration.Plugins.Contexts = Game: PhysicsKit + OtherKit + YetAnotherKit

The code generator will pick this up and when generating extensions for the Game context it will load and inspect the PhysicsKit and also generate extensions. This works great. You can now write sth like this

var context = new GameContext();
var e = context.CreateEntity();
// SomeComponent from the Game context
e.Some();

// Generated extensions from the Physics Kit. The Game context and the GameEntity can now also use the components from the Kit
e.AddPhysicsKitPosition(4, 5, 6);
e.AddPhysicsKitVelocity(10, 10, 10);

Console.WriteLine("e: " + e);

Console.WriteLine("e.position.x: " + e.position.x);
Console.WriteLine("e.physicsKitPosition.x: " + e.physicsKitPosition.x);

The code generator produces a few new interfaces as well.
PhysicsKitEntity will now implement the new IPhysicsKit interface
PhysicsKitContext will now implement the new IPhysicsKitContexinterface
PhysicsKitMatcher will now implement the new IPhysicsKitMatcherinterface

Matchers are different now, you access them through the context:

context.matcher.physicsKitPosition

This is necessary because in the scope of the PhysicsKit, the PositionComponent might have index 0, but when used as a Kit the index gets remapped ontop of already existing indices and will be another index. Since context.matcher will return an IPhysicsMatcher, it works both in the Kit and in the Game. In the Kit IPhysicsMatcher will be the actual PhysicsMatcher , in the game it will be MyGameMatcher which implements IPhysicsMatcher

This is what I called Phase I. Works great. Done. But:

Phase II is about also using systems from another Kit. So I added a simple MoveSystem to the PhysicsKit project

using Entitas;

namespace PhysicsKit {

    public sealed class MoveSystem : IExecuteSystem {

        readonly IGroup<PhysicsKitEntity> _group;

        public MoveSystem(IPhysicsKitContext context) {
            _group = context.GetGroup(context.matcher.AllOf(
                context.matcher.physicsKitPosition,
                context.matcher.physicsKitVelocity
            ));
        }

        public void Execute() {
            foreach (var e in _group) {
                var p = e.physicsKitPosition;
                var v = e.physicsKitVelocity;
                e.ReplacePhysicsKitPosition(p.x + v.x, p.y + v.y, p.z + v.z);
            }
        }
    }
}

Going back to the MyGame project and using it like this would be the goal

var system = new MoveSystem(context);

The problem:

My current problem is to resolve the casting problems. In theory MyGame context, entity and matchers will implements IPhysicsContext, IPhysicsEntity, IPhysicsMatcher.

Problem 1:
MyGameContexts implements IPhysicsContexts
This was my 1st try in order to pass it to the MoveSystem which requires a IPhysicsContexts in the ctor. But then MyGameContext has to implement all the methods from the IContext<T> interface again, now with T being PhysicsKit additionally to MyGame. This will result in duplicated methods.
E.g. the IContext<T> interface makes you implement TEntity CreateEntity(). By creating a new context new Context<GameEntity> this is already taken care of, but when now implementing IPhysicsKit I need to implement PhysicsKitEntity CreateEntity(). Yet another Kit would require to implement SomeOtherKitEntity CreateEntity()
But obviously I cannot cast and GameEntity to a PhysicsEntity.

I see, the text is already pretty long, so I guess I'll stop here ;)
The short version: I tried casting and implementing different interfaces to resolve the issue, but I'm not there yet. I'll probably have to step back and check if there are other solution or try again on the weekend.
If you guys have any suggestions or ideas how to solve this, let me know!

[sschmid]

EntitasKitsTest.zip

Here's the project if anyone want to see the setup. Pretty simple. EntitasKitsTest.sln contains boths projects. Goal: Get rid of compiler errors :) Or find a completely new approach. Go and check the generated code and maybe modify it, e.g. try what happens if GameContext also implements IPhysicsContext...

I'm currently trying to find out what the generated code should look like. So main objective is to mess with the generated code.

[jgj]

Seems to me if I'm going through the trouble of telling the code generator what context I want the kit to exist in that the code generator should generate code to integrate it transparently into said context. As an end user of a kit, do I really need to make the distinction at compile-time that a set of components/systems came from a kit?

It makes sense to me that the code generator would generate versions of the kit's systems/components that function/appear the same as systems/components I've authored myself. Otherwise I'm taking on additional mental overhead to use a kit, which is contrary to the reason I'm using a kit in the first place.

Perhaps that means creating kit-authoring-specific types (Context, Entity, Matcher, etc) that can function for testing/building a kit and easily be replaced by the code generator with the end user's types?

[ethankennerly]

Do any of you know tips on how to reuse:

1. a reactive system in two different game repos?

I tried to make a few Entitas components, systems, and MonoBehaviours that I could reuse between games. I ran into problems that are related to those discussed above.

1. The examples of a reactive system hard-code:
   1. the entity class, such as GameEntity.
   2. the context class, such as GameContext.
   3. the context property on the generated contexts object, such as contexts.game.

I can generalize to TEntity and TContext. Yet the generic system looks verbose and less useful. With a generic class I cannot operate on components, unless those components have an interface, which is generated if the components exist in multiple contexts.

Or I can specify the entity and context. Then the code in the library is non-compatible with the code in the specific game, because the library code needs to be generated to function, but the specific game code also needs to generate code, which goes into the specific game's folder. So the library is always in a broken state. It can't be tested outside of a game.

Namespacing is important to write library code. I'm probably doing it wrong, but last I tried, namespaces made awkwardly verbose generated code by prefixing the namespace to the component variables in the context.

[c0ffeeartc]

Right now Kit feature blockers are:

• partial keyword in generated folder
• ComponentsLookup indexes
• Contexts facade coupling

Solutions could be:

• Extension methods
• Attribute/reflection during initialization (similar to PostConstructorAttribute in Contexts)
• static int initialization for indexes

#407 Entitas.Generic solved these in one way
Could you tell if any changes in this field are expected?

[JesseTG]

What if Entitas adopts a standard set of contexts (but continues to allow new ones)?

[JesseTG]

Actually, wait, here's a better idea; what if instead of directly distributing custom systems or components, custom code generators that generate these systems or components are shared instead? The code generators, to my knowledge, don't care how Entitas is implemented, so this wouldn't require a fundamental rework of Entitas. Instead, this would require more documentation and guidance for custom code generators. And, just as importantly, this would require the code generator implementation process to be more streamlined, with less boilerplate required by the user.

[FNGgames]

I was thinking about a code gen oriented solution to kits recently.

[JesseTG]

Do tell.

[FNGgames]

I'm not super advanced in my thinking here, but as Jesse said, the code gen are pretty much walled off from your specific implementation. The idea would be to add a host of new attributes for kits, that would generate things. You could have the cg generate components and systems and even contexts, and the user could have a bit more custom control over the naming / prefixes / suffixes etc with some configs.

[JesseTG]

This means that instead of talking about how to implement kits, we can now talk about how to improve the usability and documentation of custom code generators. What do you think, @sschmid?

[SanderMertens]

@sschmid the module/packs/kits idea is something I implemented with flecs modules, where just like you describe, there are a number of reusable modules that implement a feature, like rendering, collision detection, movement etc. Some of the projects created with these modules are this, this and this.

I'll deposit some ideas on challenges I had to overcome here, in case it's useful for the Entitas design:

Ordering
One major challenge I had to address was how to order systems the right way. Some systems imported from a module should be executed at the beginning of a frame (like cleanup systems), while others needed to be ran after game logic was processed (collision detection). I addressed this with three mechanisms:

• Systems can be assigned to a "phase" (PreUpdate, OnUpdate, PostUpdate, Render etc)
• Within a phase systems are executed in the order of declaration
• Modules can import other modules (requires allowing for re-importing modules)

The guiding principle is: there should be no direct dependencies between systems in different modules, but there can be dependencies between modules. So far this seems to work, though I'm still refining the different phases. In Entitas this may be easier, as you could use the Unity execution order.

Module organization
A second challenge was how to organize modules. I ended up creating separate modules for components and systems. The component modules establish a common interface for doing certain things, like expressing physics, geometry, or even something completely different like an HTTP server. The system modules only implement the systems that use these components. This lets you "easily" plug in different implementations for the same functionality. For example, an application could swap to a different rendering engine by just importing a different rendering system module, while still using the same components module.

Performance degradation because of redundant systems
A third challenge was how to prevent systems that don't do anything from impacting performance / RAM footprint. When importing a systems module there is a good chance that a number of systems won't be used. In a normal application that's ok, since the user has full control, but in a module a user may not have access to the systems. I addressed this in three ways:

• The ECS framework automatically removes systems that don't match any entities from the critical path
• The application can pass a flags argument into the import statement to include/exclude certain features (for example, I want to import physics, but only for 2D)
• The import function returns a struct with handles to the imported systems, which an application can then manually turn on/off

UI clutter
The fourth challenge I faced was more UI related. I have a web dashboard that shows all the loaded systems (similar to the Entitas panel in Unity) and when you import multiple modules, you can get a lot of systems in your UI. Even for a trivial Pong example, I had over 50 systems, most of which weren't doing much. To address this, I added an EcsHidden tag to the framework which can be used to mark "framework" functionality that is not directly related to the application logic. An UI can use this information to automatically hide or flag these systems, so that a user can easily see which systems are "application logic", like this (orange triangles are framework):

Anyway, that's a blurb with some thoughts, hope it is helpful :)

[sschmid]

I haven't forgotten about this ancient issue :D
We might finally get there soon with the new code generator #1005