Tracking issue: sync/harvest speed #1168
Labels
area:backend
Everything backend related
area:sync
Syncing with an Opencast instance
kind:improvement
priority:low
Low priority
Since I looked into this briefly, let me just post some numbers here.
The bern numbers are from a resync today. 80min were spent on Opencast side, 10min on Tobira side. For Tobira, storing series seems to be much slower. 500 series take roughly 14s to store. 500 events only take 1s. It's the opposite on the Opencast side: preparing a response with 500 series takes roughly 1s, but preparing one with 500 events takes 35s.
Others can use these numbers to estimate how long the sync is going to take on their system.
And for us: it would be nice to optimize these sync times still. Like, this is super slow compared to the data we are processing. This could be made faster by orders of magnitude. However, the more important question is: how much faster can we get it with only investing a little bit of development time? ;-) I still guess that a few hours invested here could bring significant wins. But yeah, not our priority.
The text was updated successfully, but these errors were encountered: