minimal snapshot #2904
Open
minimal snapshot #2904
+1,260
−59
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This special case is not needed as `big.NewInt(0).Bytes()` already returns empty []byte start key. This reverts commit 898d5c3.
magicxyyz
requested review from
eljobe,
tsahee and
ganeshvanahalli
and removed request for
eljobe and
tsahee
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves NIT-3057
This PR adds database snapshotter that allows exporting a minimal execution database (here called
snapshot) that consists of:In connection with manual copying of consensus database (
arbitrumdata) this can be used to create a complete databases snapshot for a full node.The snapshotter can be enabled with flag:
--execution.database-snaphotter.enable(disabled by default).Currently there are two ways to trigger the snapshot export:
SIGUSR2to nitro process - starts the export with snapshot head set to latest block which state is found in disk database (e.g.docker kill -s "SIGUSR2" nitro-node)snapshotter_snapshotrpc method that takes block number or block tag parameter - starts the export with snapshot head set to latest block that is not newer then specified block and has state available in disk database;snapshotter_resultmight be called to check the export status (the method takes "rewind" boolean parameter if set true and previous snapshotter run finished it discards cached result and new snapshot can be started)Initial version supports exporting only to geth format database. The output database is specified by following options:
--execution.database-snapshotter.geth-exporter.output.datadirectory of stored chain state (default "snapshot")
--execution.database-snapshotter.geth-exporter.output.ancientdirectory of ancient where the chain freezer can be opened (default "ancient")
--execution.database-snapshotter.geth-exporter.output.cachethe capacity(in megabytes) of the data caching (default 2048)
--execution.database-snapshotter.geth-exporter.output.db-enginebacking database implementation to use ('leveldb' or 'pebble')
--execution.database-snapshotter.geth-exporter.output.handlesnumber of files to be open simultaneously (default 512)
--execution.database-snapshotter.geth-exporter.output.namespacemetrics namespace (default "l2chaindata_export")
--execution.database-snapshotter.geth-exporter.output.pebble.*extra options for pebble engine, see --help for details
--execution.database-snapshotter.geth-exporter.ideal-batch-sizeideal write batch size in bytes (default 104857600)
Number of threads used by the snapshotter is specified by
--execution.database-snapshoter.threadsoption.Pulls: OffchainLabs/go-ethereum#402