Orleans grain persistency background

[koenbeuk]

Hey All, I've been using Orleans on and off for various projects for a while now (since V1). Over the years I've come to get more familiar with how to design grains and how to work with persistence of grains. Looking at the docs though, it's talking about using the IPersistentState<TState> interface whereas inheriting from Grain<TState>. is considered a legacy feature.

While I can name some of the advantages of using IPersistentState<TState>, e.g.:

• No restricting on inheriting from a base class
• Multiple persistent states per grain

I fail to see why it would make a Grain<TState> a legacy feature. In 95% of my working examples, the above advantages are not needed. The majority of my grains have some facts in common (which again, may be unique to my situation):

• The grain has 0 to 1 state at most.
• The grain uses the default provider
• The grain would persist operational state. (Such as has 'indexing' been scheduled for this grain or some temporary computed cache).
• The grain would still manually interact with external databases to query more advanced information.

As for clarification:

class UserGrain : Grain<UserState> { }

now becomes:

class UserGrain : Grain { 
 private readonly IPersistentState<UserState> _state;
  public UserGrain([PersistentState("user")] IPersistentState<UserState> state) {
    _state = state;
  }
}

I'm sure there have been discussions about this before but I was not able to find them. Meanwhile the documentation is not clear about the reasoning for staying away from Grain<TState>. There are mentions about it here https://dotnet.github.io/orleans/docs/grains/grain_persistence/index.html and here https://dotnet.github.io/orleans/docs/resources/frequently_asked_questions.html

So I would like to ask these questions to the community:

• What are the reasons for moving away from the base class to the interface approach?
• Where can I read up on earlier discussions?

Thanks!

[JorgeCandeias]

The most beneficial consequence I've seen in production code is ease of unit testing. Being able to inject explicit state into the constructor makes it trivial to mock grain persistence data. Whereas being dependent on implicit state by convention requires developers to set up the testing host and worry about shared state in the test runs. Granted, it's more reliable, but it also takes longer to both write and run tests and make sure nothing clashes.

In some cases, I've also drawn benefit from the ability to have multiple state items per grain, where each data state has different granularity and change frequency, even though they belong in the same aggregate. This is valuable when milliseconds make a difference to the application. You'd get the same benefit from using a database, of course, but it is still useful if you want the benefit without the extra database coding.

[koenbeuk]

@JorgeCandeias

Unit testing is an interesting point. I personally have never had the need to test without a test cluster as most of my grains are somewhat tied to other grains or features within Orleans but I can indeed see the advantages.

Multiple persistent states are also an advantage. I would argue in favour of keeping these features side-by-side with the ability to combine them:

public class UserGrain : Grain<UserState> {
    public UserGrain([PersistedState("profile")] IPersistedState<ProfileState> profileState) { ... }
}

Typed grain classes are concise and great for beginners and cover most use cases in my experience. In order to move this discussion forward, I want to suggest a few changes that we make in Orleans and especially the docs:

1. Don't mark Grain<TState> as a legacy feature and keep it together with IPersistedState<TState> as a way to interact with State in Orleans.

2. Document the use of Grain<TState> before introducing IPersistedState<TState> as the former is easier to understand, as it requires less code as demonstrated in my initial post.

3. Update the documentation to use the Grain<TState> with the default provider, thereby delaying the introduction of the StorageProvider attribute.

4. Make the name argument of the PersistedState attribute optional. So instead of having to type: [PersistedState("state")], you can now simply type [PersistedState] In the following example, it should be clear that our persisted states should be named differently and the names that we expect are userState and profileState:

public User([PersistedState]IPersistedState<User> userState, [PersistedState]IPersistedState<Profile> profileState) { .... }

(Reflection or even source generators can get us the argument names for these different state arguments)

5. Continuing with the above, we can make the [PersistedState] attribute optional since you only need it if you want to change the name of the State or use a non-default provider.

Looking forward to your thoughts

[JorgeCandeias]

If I recall from past comments I think the move to IPersistentState follows the general trend in dotnet of preferring flexible injection over limiting inheritance. This trend started long ago with the normalization of the host pattern and native dependency injection for just about every application. @ReubenBond can comment on this better though.

[koenbeuk]

Orleans did not mark the Grain class as legacy nor do I suspect it can do so as a big part of the design is based around features provided by the Grain base class such as lifecycle methods (activate/deactivate), Identity, GrainFactory, StreamProvider, Lifetime control etc. I'm in favor of providing an out of the box model so that I can narrowly implement a Grain class that only implements the lifecycle methods it needs and uses DI to reference the GrainFactory and GrainIdentity and such. This however is not a trivial task and would probably require a major version bump.

Meanwhile, we're stuck with the Grain base class (which is not a bad thing). Since this is not going away any time soon, I would vote for keeping the current Grain<TState> in place as well. Newcomers can this way learn that the primary way of interacting with Orleans from a grain is through a base class. When we move to an out-the-box experience with interfaces and proper DI, we can declare both Grain and Grain<T> as legacy.

[SebastianStehle]

I don't see that the grain class is needed anymore. It would better if you could just implement a POCO class and inject the IGrainActivationContext interface.

I really like the proto actor model for persistence: https://proto.actor/docs/persistence/. At the moment the persistency system is very limited and provides very little value. You can just rebuild your own system with a few lines of code.