-
Notifications
You must be signed in to change notification settings - Fork 429
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Question: state schema path & pebble db #2323
Comments
This would be incredibly useful |
I do not believe this is possible. I have seen |
I suppose it's related to #2324 |
It is now possible to launch with: /usr/local/bin/nitro ... \
--execution.caching.state-scheme path \
--persistent.db-engine pebble However, I am unable to bootstrap a node like this. The existing genesis snapshot from https://snapshot.arbitrum.foundation/arb1/nitro-genesis.tar contains leveldb information, of course, so
Without a pebbledb genesis snapshot, I presume it's not yet possible to use the new state scheme in production. So, I would echo @Zorato 's request for pebble-backed snapshots. Side note: I looked into low-level conversion from leveldb to rocksdb or pebble but this didn't seem trivial. However, if it worked, then existing snapshots could be re-used. |
Correct, because of the historical data, this isn’t trivial. Optimism had a similar challenge. Team-provided pebble/path snapshots would be the solution. |
That’s less interesting than path tbh. Sure, PebbleDB is better supported, shuts down faster, and is less likely to corrupt. These are all desirable and reasons to use Pebble. A conversion of the existing DB would then be Pebble / hash, and you’re still stuck with manual offline pruning every so often. It’s path that introduces continuous pruning. That’s the one that has the biggest operational impact. And yes sure, you may as well go with PebbleDB if a resync is inevitable anyway. Fingers crossed the team can provide a historical snapshot in Pebble/path form. |
Good point. A thought I had last night is that the testing code contributed in #2324 probably contains a good portion of the work required to stand up the new pebble/path state store. A utility script could be derived from the test code that inits the empty pebble/path database, iterates the genesis snapshot block-by-block, and writes to the pebble/path db. From there, wouldn't it be a matter of syncing to a live archive node, as usual? Once synced, prune the state store and make a tarball. Easier said than done, I'm sure, but I am watching with great interest. edit: removed user tag |
I just noticed part of this problem has been solved in #2061 - specifically converting leveldb to pebbledb. I do not think this this tool provides conversion from hash to path state store. However, I did notice some testing code related to Path: https://github.com/OffchainLabs/nitro/pull/2061/files#diff-f6a86af8229f2ce0d04e0a35bb0a8caab9a94de92a36d85451303fbc2b560d30R100 |
Is there any chance to get pruned snapshot of Arbitrum One node with PebbleDB engine (instead of LevelDB) with state.schema=path (instead of default one path)?
Pruning node every few weeks causes downtimes and requires lots of free disk space to maintain, while state schema path may resolve growing chaindata size issue.
Any advice on how to sync node from scratch with these parameters are appreciated.
The text was updated successfully, but these errors were encountered: