Add block quarantine to ForkedChain

[jangko]

fix #3056

[arnetheduck]

minilru is probably the better choice here, which should simplify the addOrphan logic (and perform better)

[jangko]

a079c86
requires status-im/nim-minilru#3

[arnetheduck]

we will probably need to introduce an async queue or a similar construct to process quarantine blocks - if we proces 128 blocks in one async time slice, that will likely disconnect everyone due to the long "downtime"..

in the meantime, I think it's enough that we keep 32:ish blocks here, that's already 6 minutes of block time.

[advaita-saha]

Yeah agreed, mostly the sync lags by maximum of 10 blocks
32 seems like a more reasonable number to keep

[jangko]

After I rethink about it, the block quarantine also overlap with importHeader functionality, especially when importing blocks to FC. They can share the same async worker. And somehow related to #3002

[arnetheduck]

also overlap with - not necessarily, ie it doesn't need headers to perform its work - when the normal syncer reaches a point here quarantine blocks exist, there's no need to separately download headers since the blocks in the quarantine already have headers too.

[arnetheduck]

ie I think the quarantine, even just as is, is a step forward towards stable sync

[advaita-saha]

Currently the block quarantine implementation has no effect on improving sync since,

when a block arrives whose parent is unknown, it is put in a bounded quarantine

Here the arrival of the block means also means payload coming from engineAPI. Thus when a payload with unknown parent hash comes from the engineAPI call newPayload we should put it in the block quaratine, to be later used to complete the remaining ~10 blocks sync. Not happening in the current impl

So in the current implementation when a payload comes to us, with a unknow parent we are doing this.

nimbus-eth1/execution_chain/beacon/api_handler/api_newpayload.nim

Lines 174 to 181 in 44db666

	# If the parent is missing, we - in theory - could trigger a sync, but that
	# would also entail a reorg. That is problematic if multiple sibling blocks
	# are being fed to us, and even moreso, if some semi-distant uncle shortens
	# our live chain. As such, payload execution will not permit reorgs and thus
	# will not trigger a sync cycle. That is fine though, if we get a fork choice
	# update after legit payload executions.
	let parent = ben.chain.headerByHash(header.parentHash).valueOr:
	return ben.delayPayloadImport(header)

And currently we are only using the quarantine when we are trying to import a block into forked chain, which is fine. But in all the payloads we received with missing parent, are never added to the quarantine so in real life scenario never helps us

nimbus-eth1/execution_chain/core/chain/forked_chain.nim

Lines 523 to 545 in b3150ec

	while c.quarantine.hasOrphans():
	let orphan = c.quarantine.popOrphan(parentHash).valueOr:
	break
	c.hashToBlock.withValue(parentHash, parentCandidatePos) do:
	parentHash = c.validateBlock(parentCandidatePos[], orphan).valueOr:
	# Silent?
	# We don't return error here because the import is still ok()
	# but the quarantined blocks may not linked
	break
	do:
	break
	do:
	# If it's parent is an invalid block
	# there is no hope the descendant is valid
	debug "Parent block not found",
	blockHash = header.blockHash.short,
	parentHash = header.parentHash.short
	# Put into quarantine and hope we receive the parent block
	c.quarantine.addOrphan(blk)
	return err("Block is not part of valid chain")

Ideally when the sync is happening we are getting a lot of newPayload and fCU calls from the CL. Now these payloads are possible future parts of the chain when we are syncing older blocks. And we want to use these received payloads to complete the sync if possible i.e validity checked. This helps the syncer not lag behind by 7-10 blocks every sync cycle
That being said, it implies also using the block quarantine before calling the syncer in fCU call and newPayload with unknow parent to check we can construct the chain