[Proposal] Resource Override

[tjhancocks]

Aspect	Information
Author	Tom Hancocks
Proposal Date	11 December 2019
Status	Review

Proposal Outline

KDL currently allows only for the creation of new resources, and has no concept of overriding existing resources. Whilst it is possible to override them by simply creating a new resource, if the new resource only needs to change the name, or a single value, then it can be tedious to have to define each of the fields with the original resources values.

Detail

At present there is only one way to define a resource in KDL.

declare Resource {
    new (id = #128, name = "Replacement Shuttle") {
        ` specify all of the fields for the original "Shuttle" and change the values you wish to alter.
    }
}

This is a passable solution and it gets the job done, but it is not intuitive/friendly to the developer. It should be possible for KDL to explicitly state that a declaration/definition is overriding another resource.

declare Resource {
    override (id = #128, name = "Ripoff Shuttle") {
        ` specify only the fields that need to alter.
        cost = 100000;
    }
}

This would allow the assembler to know that the resource should be overridden, and require it to load the existing resource data and use that as a basis for the new resource. This will however, require a number of additions and improvements to the assembler.

• The assembler needs to be able to load other Resource Files to use as context for the plugin being assembled.
• The assembler needs to be aware of Kestrel and where it keeps scenarios on disk.

These additions are the minimum required to allow this to work.

[Thunderforge]

Say that we added the ability to override a Resource with id #128 (as in the second snippet). Given that it would be the preferred way of doing it, what would be the desired behavior of then creating a new Resource with the same id as an existing Resource (as in the first snippet)?

My guess is that the Kestrel engine would be unable to handle two Resources with the same id and would throw a runtime error. So ideally we'd want to detect that issue as early as possible by throwing a compile-time error if the first snippet was run. If so, that's an assembler behavior we'd want to implement.

The assembler needs to be able to load other Resource Files to use as context for the plugin being assembled.

Two solutions come to my mind for this:

• A new flag (e.g. -r for resource file)
• A manifest file of some sort that provides paths to all the needed resources, and then it's passed into the command line (e.g. -m for manifest file).

The assembler needs to be aware of Kestrel and where it keeps scenarios on disk.

I don't think that there should be a standard location for scenarios on the hard drive, since that could vary from platform to platform, and there could be a variety of reasons why they could be in different locations from machine to machine.

The two solutions above would be the same for this problem. I suspect that the manifest file would be the better solution since then we have one flag that's passing in data about potentially many scenarios and other resource files (and in the future, could be extended for things like media resources). Given the potential scope of that, it may be worth tracking in a separate issue.

[tjhancocks]

Yeah I think a conflict would raise an ID collision error. Kestrel won’t throw an error upon finding duplicate IDs in a single resource file, but it would be undefined behaviour. But kestrel should need to have to know about such possibilities, thus the assembler throws an error.

Definitely agree with a manifest for all files in your current project. Not sure what that should like though. 🤔

As for scenario locations, I think these should baked into the assembler, based on platform being compiled for. Kestrel won’t be moving these frequently (if ever), and it’s exactly what it does. We can also add options for overriding this location.

I need to get some documentation/plans put together on the overall requirements from Kestrel, and ideas I’ve had for KDK.

_{Sent with GitHawk}

[Thunderforge]

Definitely agree with a manifest for all files in your current project. Not sure what that should like though. 🤔

I have some ideas for that based on other projects I've worked on. When I get a chance, I can create a proposal issue for it.

As for scenario locations, I think these should baked into the assembler, based on platform being compiled for. Kestrel won’t be moving these frequently (if ever), and it’s exactly what it does. We can also add options for overriding this location.

I'm not sure I understand why baked in would be preferable to user-defined. Assuming scenario means "base game", wouldn't that be in any number of locations as more games than EVO are supported? On macOS, I assume it would be in something like:

• Applications/EV Override Remaster.app/Contents/Resources/Scenario.kdt
• Applications/EV Nova Remaster.app/Contents/Resources/Scenario.kdt
• ~/Documents/My Kestrel Plugins/Foobar Total Conversion Scenario.kdt

[tjhancocks]

Hmm 🤔.

Good point 👍. This is where a second set of eyes makes sense. I was thinking about it purely in terms of how Kestrel currently is, which is to have all scenarios defined in a common location: ~/Library/Application Support/Kestrel/Scenarios/Foobar.kestrel-scenario, but ultimately this is defined on a per game basis.

Sorry, this is me being in too deep to the current state of Kestrel to see the wider picture 😅

[andrews05]

Maybe an environment variable like EV Nova currently uses?

[tjhancocks]

So that method of overriding is already intrinsically present in the ResourceFile system, and is actually the domain of the Kestrel, not the development kit. This is the other side of the equation, when we assemble the resources.

For example, say we have:

declare Foo {
    new (id = #150) { ... }
    override (id = #150) { ... }
}

Which one should it take? Should just take the most recent, and override the scenario resource? Should it override the newly created resource? Or should it just throw an error?

It's probably important to mention that override in this context means something different to what it does in Kestrel. It's a subtle difference, but an important one. We're redefining a resource when we override it.

It we try to override a resource that doesn't exist, then this is an error, because it will not have the initial source data from which to build a new resource.

Image a Foo resource. It has two integers, bar, baz. Encoded as binary they look like this:

declare Foo {
    new (id = #231) {
        bar = 180;
        baz = 512;
    }
}

0x0000 |  00 80 02 00

When we override we're asking the assembler to take all of the initial data and substitute only what we need, and not have to redefine everything, like so:

declare Foo {
    override (id = #231) {
        baz = 255;
    }
}

Which results in:

0x0000 |  00 80 00 FF

Sorry if I explained bits you already know/understood, but this might be helpful to others coming in who don't.

[andrews05]

Right, well my expected behaviour would be that the most recently declared definition of the resource would have precedence, starting with the new one in the current file and working backwards through the manifest (or however the other files get defined).

As an extension of this proposal, a means of easily creating multiple variants of a resource would also be extremely handy. Something like:

declare Resource {
    new (id = #129, name = "Ripoff Shuttle", extend = #128) {
        ` specify only the fields that need to alter.
        cost = 100000;
    }
}

Where #128 is duplicated to #129 with a modified cost.
(This could even be used for overriding by specifying the same value for both id and extend)