Collection notifications without deep change checking? (Performance optimization)

[aehlke]

Problem

When I use .collectionPublisher even though I am mapping to void and debouncing, Realm spends a LOT of CPU time in DeepChangeChecker. I don't actually need to know what changed as long as I can run some code anytime anything changes for a given object type.

This performance is so severely bad that I will need to stop using Realm (earlier than I would like to transition away from it)

Solution

Is the DeepChangeChecker behavior necessary when I'm discarding the results of collectionPublisher? Is there a way I could create a custom notifier (in a Realm/RealmSwift fork or not) which skips the DeepChangeChecker work? I haven't been able to fully understand the notifications code enough yet.

Alternatives

No response

How important is this improvement for you?

Dealbreaker

Feature would mainly be used with

Local Database only

[aehlke]

@nirinchev sorry to ping - perhaps you know a lead on this? Thank you

[tgoyne]

The keyPaths parameter can be used to control what things are checked. keyPaths: [] will report only modifications to the object itself (i.e. if the object is deleted). keyPaths: ["foo", "bar"] will report changes to the foo and bar properties only, but not perform deep checking on those properties. I believe we never exposed keyPaths: ["*"] for all shallow properties in Swift, but you can work around it with something like keyPaths: ObjectType.schema.properties.map(\.name) (this specific construct is very inefficient and you'd probably want to cache the array if you'd ever call it more than once).

[aehlke]

@tgoyne thanks for chiming in despite being reassigned off this...

I have tried keyPaths but saw similarly expensive DeepChangeChecker usage - it was slow to compute which key paths changed to determine what to filter. I will need to re-confirm that

I see now (though without using keyPaths anymore after discovering the above) that the expensive part of DeepChangeChecker is check_outgoing_links so I'm removing all relations and replacing them with IDs to see if that resolves enough of this issue. But this may still be insufficient given your comment if my prior testing with keyPaths was also too slow (I hadn't realized at the time that keyPaths can prevent the check_outgoing_links checks too)

edit: Re-reading Realm source I believe it was CollectionKeyPathChangeChecker which was also too slow when I was using keyPaths. So it may have achieved shallow checking to avoid check_outgoing_links but introduced a new bottlenecks. So I will persevere with trying out removing all relations so that I can avoid both CollectionKeyPathChangeChecker AND check_outgoing_links