Skip to content
This repository has been archived by the owner on Jul 20, 2024. It is now read-only.

a few questions about hlc implementation details #1

Open
uptonking opened this issue Jan 16, 2023 · 4 comments
Open

a few questions about hlc implementation details #1

uptonking opened this issue Jan 16, 2023 · 4 comments

Comments

@uptonking
Copy link

uptonking commented Jan 16, 2023

  • I really love your tinybase HLC implementation and try to play with it. I got a few questions:

prerequisite: every data store has a change-tree attached to it, which contains the store's full changes-data/operations.

🤔 Q1: when building a store-change-tree in addLeaf, why is HLC splitted into 4 parts?

{ time1: { time2: {counter: {clientId:changeObj} } } }

  • why not just keep 3 parts { time: {counter: {clientId:changeObj} } } ? or why not split into more parts by add time3/time4 ?
  • why is time's 7-digit splitted to 3+4? not 4+3 / 1+6 / 2+5?
  • I'm curious about the design.

🤔 Q2: in your sync example, it seems that for every sync logic,a whole tree must be sent to another.

  • It seems crazy if one made a lot of changes, then every following sync needs to send a new change-tree containing all previous data?

  • I have an idea1 to use a central server and keep a change-tree there, then client just sends current change-operation/data instead of full tree . Then it's the server rather than the client to build the change-tree and return the minimal diff tree to the client.

  • after a good sleep, I have an idea2 to split every sync into 2 phases: sendMeta + sendChanges. In sendMeta, still send a change-tree but without change content. In sendChanges, minimal diff changes content will be sent.

  • Do you have any idea to solve this?

🤔 Q3: in handleChange, latestHlcsByCell seems useless.

Is It safe to remove it?

@jamesgpearce
Copy link
Contributor

why is time's 7-digit splitted to 3+4? not 4+3 / 1+6 / 2+5?

Because this will get serialized into a patricia trie, I wanted to minimize how much that got sharded and I figured that a human is likely to use an app for a few hours at a time, not solid days or only minutes. Otherwise, no hard rule here!

@jamesgpearce
Copy link
Contributor

it seems that for every sync logic,a whole tree must be sent to another.

Yes, that's right.

A true offline/local first implementation should in theory support someone coming back online a year after they made a change

I think to scale we need some sort of "changes before this time will be ignored" baseline. It might be possible to do that on a table-by-table or row-by-row basis, I'm not sure

@jamesgpearce
Copy link
Contributor

latestHlcsByCell seems useless.

Isn't that the part that does LWW?

@uptonking
Copy link
Author

latestHlcsByCell seems useless.

Isn't that the part that does LWW?

  • lww only happens in setChanges

  • latestHlcsByCell is only used to get the hlc for one change. and 3-level mapGet is a little verbose.

  • maybe a tinybase store can be used to persist all changes hlcs

    • then we can use hlcsStore.get(id) to get the latestHlcByCell, which is more intuitive

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants