Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bridges V2 refactoring backport and pallet_bridge_messages simplifications #4935

Merged
merged 64 commits into from
Jul 12, 2024

Conversation

bkontur
Copy link
Contributor

@bkontur bkontur commented Jul 3, 2024

Summary

This PR contains migrated code from the Bridges V2 branch from the old parity-bridges-common repo.
Even though the PR looks large, it does not (or should not) contain any significant changes (also not relevant for audit).
This PR is a requirement for permissionless lanes, as they were implemented on top of these changes.

TODO

Relates to: paritytech/parity-bridges-common#2976
Relates to: paritytech/parity-bridges-common#2451

serban300 and others added 29 commits June 26, 2024 21:08
* Define Chain::STATE_VERSION

* Add vec_db module

* Use VecDb instead of StorageProof for message delivery

* Make sure that the TrustedVecDb is sorted

* Address review comments

* Run benchmarks on parent commit

* Run benchmarks with new code

* Fix test

* Fix code review comments
* moved FromBridgedChainMessagesDeliveryProof to bp-messages

* spelling

* fix benchmarks compilation
* moved FromBridgedChainMessagesProof to bp-messages

* fmt
* moved message files to separate folder

* ...and fix paths in code

* fmt

rebase nit
* Adjust messages pallet benchmarks

* Address comment
* moved files around

* and fix compilation

Move Chain::ID from relay-level Chain to primitives-level Chain (#2181)

* move Chain::ID from relay-level Chain to primitives-level Chain

* removed chain IDs from bp-runtime

* add missing file header

Adjust weights (#2185)

fix clippy (#2186)

Remove source header chain (#2183)

* moved messages proof verification to messages pallet

* removed SourceHeaderChain

* cleanup

* benchmarks compilation

* fix benchmark tests compilation

* clippy

* fmt

Use compact proofs for messages delivery confirmation (#2187)

* Use compact proofs for messages delivery confirmation

* Fix benchmarks

Move messages delivery proof verification to pallet (#2189)

* rename messages_proof.rs to proofs.rs

* moved delivery proof verification to the messages pallet

* removed TargetHeaderChain

* cleaning up

* fixed benchmarks compilation

* Update modules/messages/README.md

Co-authored-by: Adrian Catangiu <adrian@parity.io>

* uncommented test and removed printlns

* vec![].into_iter().collect() -> vec![].into()

---------

Co-authored-by: Adrian Catangiu <adrian@parity.io>

removed MaxUnrewardedRelayerEntriesAtInboundLane and MaxUnconfirmedMessagesAtInboundLane from messages pallet config (#2190)

* removed MaxUnrewardedRelayerEntriesAtInboundLane and MaxUnconfirmedMessagesAtInboundLane from messages pallet config

* fixed doc

* fixed benchmarking code

* more fixes

remove MaximalOutboundPayloadSize and use BridgedChain::maximal_incomging_message_size instead (#2191)

remove MessageBridge trait (#2192)
* Use compact proofs for parachains

* Remove StorageProofChecker

* Cleanup

Reuse messages proof generation from messages pallet (#2195)

* reuse messages proof generation from messages pallet

* incorrect merge

fixed remaining TODOs after messages pallet Config refactoring (#2196)

Return UntrustedVecDb from prove_storage() (#2197)

Storage proofs related renamings (#2198)

* Storage proofs related renamings

* Leftovers

* StorageSize -> StorageProofSize
* add test for maximal message dispatch weight

* fixed weight computations - use kb for both size-related benchmarks

* fix typo in weight formula

* decreased maximal message size to 64Kb
* Box messages proof argument in receive_messages_proof call

* fix benchmark tests
* StorageProofSize -> StorageSize

* Rename benchmarks

* StorageSize -> UnverifiedStorageProofParams

* Fix clippy

* Fix priority boost
* update async-trait to 0.1.69

* more clippy fixes
* some code grooming: enable warn(missing_docs) for all piblic crates + added missing documentation + removed obsolete clippy/deny workarounds

* removed strange allow + added comment related to other allow

* removed incorrect_clone_impl_on_copy_type which is unknown to CI clippy
…=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_relayers

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_relayers

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_parachains

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_parachains

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_messages

".git/.scripts/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_messages
@bkontur bkontur added the T15-bridges This PR/Issue is related to bridges. label Jul 3, 2024
bkontur and others added 3 commits July 4, 2024 16:05
…dle (#4944)

Original PR with more context:
paritytech/parity-bridges-common#2211
Relates to:
paritytech/parity-bridges-common#2210

## TODO

- [x] fresh weighs for `pallet_bridge_messages`
- [x] add `try_state` for `pallet_bridge_messages` which checks for
unpruned messages - relates to the
[comment](paritytech/parity-bridges-common#2211 (comment))
- [x] ~prepare migration, that prunes leftovers, which would be pruned
eventually from `on_idle` the
[comment](paritytech/parity-bridges-common#2211 (comment)
can be done also by `set_storage` / `kill_storage` or with
`OnRuntimeUpgrade` implementatino when `do_try_state_for_outbound_lanes`
detects problem.

## Open question

- [ ] Do we really need `oldest_unpruned_nonce` afterwards?
- after the runtime upgrade and when `do_try_state_for_outbound_lanes`
pass, we won't need any migrations here
    - we won't even need `do_try_state_for_outbound_lanes`

---------

Signed-off-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: command-bot <>
Copy link
Contributor

@acatangiu acatangiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code changes look good!

Should do follow-up investigation on grandpa submit proof weight for bridge-hub-rococo.

@bkontur
Copy link
Contributor Author

bkontur commented Jul 11, 2024

bot clean
bot bench cumulus-bridge-hubs --runtime=bridge-hub-rococo --pallet=pallet_bridge_grandpa
bot bench cumulus-bridge-hubs --runtime=bridge-hub-westend --pallet=pallet_bridge_grandpa

@command-bot
Copy link

command-bot bot commented Jul 11, 2024

@bkontur https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6676444 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa. Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 2-69bc0eb3-a7d2-4a77-9a68-35bf29cea43c to cancel this command or bot cancel to cancel all commands in this pull request.

@command-bot
Copy link

command-bot bot commented Jul 11, 2024

@bkontur https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6676445 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa. Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 3-a676174b-7a34-4bc3-871b-24cfd744bc6b to cancel this command or bot cancel to cancel all commands in this pull request.

command-bot added 2 commits July 11, 2024 12:25
…=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa
@command-bot
Copy link

command-bot bot commented Jul 11, 2024

@bkontur Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-rococo --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6676444 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6676444/artifacts/download.

…=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa
@command-bot
Copy link

command-bot bot commented Jul 11, 2024

@bkontur Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=bridge-hub-westend --runtime_dir=bridge-hubs --target_dir=cumulus --pallet=pallet_bridge_grandpa has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6676445 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6676445/artifacts/download.

@bkontur bkontur enabled auto-merge July 11, 2024 14:16
@bkontur bkontur added this pull request to the merge queue Jul 11, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 11, 2024
@bkontur bkontur enabled auto-merge July 12, 2024 07:25
@bkontur bkontur added this pull request to the merge queue Jul 12, 2024
Merged via the queue into master with commit 1f8e448 Jul 12, 2024
155 of 160 checks passed
@bkontur bkontur deleted the bko-bridges-v2-backport-refactoring branch July 12, 2024 08:48
bkontur added a commit to paritytech/parity-bridges-common that referenced this pull request Jul 12, 2024
…k#4935 (#3022)

* Companion for paritytech/polkadot-sdk#4935

* Fix imports for `FromBridgedChainMessagesProof` / `FromBridgedChainMessagesDeliveryProof`

* Align with `polkadot-sdk`

* Align with actual `polkadot-sdk`

* Align with actual `polkadot-sdk`

* fmt

* Undienerize back to master

* Update tools/runtime-codegen
TomaszWaszczyk pushed a commit to TomaszWaszczyk/polkadot-sdk that referenced this pull request Jul 13, 2024
…cations (paritytech#4935)

## Summary

This PR contains migrated code from the Bridges V2
[branch](paritytech#4427) from the
old `parity-bridges-common`
[repo](https://github.com/paritytech/parity-bridges-common/tree/bridges-v2).
Even though the PR looks large, it does not (or should not) contain any
significant changes (also not relevant for audit).
This PR is a requirement for permissionless lanes, as they were
implemented on top of these changes.

## TODO

- [x] generate fresh weights for BridgeHubs
- [x] run `polkadot-fellows` bridges zombienet tests with actual runtime
1.2.5. or 1.2.6 to check compatibility
- ☑️ working, checked with 1.2.8 fellows BridgeHubs
- [x] run `polkadot-sdk` bridges zombienet tests
  - ☑️ with old relayer in CI (1.6.5) 
- [x] run `polkadot-sdk` bridges zombienet tests (locally) - with the
relayer based on this branch -
paritytech/parity-bridges-common#3022
- [x] check/fix relayer companion in bridges repo -
paritytech/parity-bridges-common#3022
- [x] extract pruning stuff to separate PR
paritytech#4944

Relates to:
paritytech/parity-bridges-common#2976
Relates to:
paritytech/parity-bridges-common#2451

---------

Signed-off-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Serban Iorga <serban@parity.io>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: command-bot <>
ordian added a commit that referenced this pull request Jul 15, 2024
* master: (120 commits)
  network/tx: Ban peers with tx that fail to decode (#5002)
  Try State Hook for Bounties (#4563)
  [statement-distribution] Add metrics for distributed statements in V2 (#4554)
  added sync command (#4818)
  Bridges V2 refactoring backport and `pallet_bridge_messages` simplifications (#4935)
  xcm-executor: Improve logging (#4996)
  Remove usage of `sp-std` on templates (#5001)
  fixed cmd bot commenting not working (#5000)
  Explain usage of `<T: Config>` in FRAME storage + Update parachain pallet template  (#4941)
  Expose metadata-hash feature from polkadot crate (#4886)
  Add `MAX_INSTRUCTIONS_TO_DECODE` to XCMv2 (#4978)
  add notices to the implementer's guide docs that changed for elastic scaling (#4983)
  `polkadot-parachain` simplifications and deduplications (#4916)
  Update Templates README docs (#4980)
  allow clear_origin in safe xcm builder (#4777)
  litep2p/peerstore: Fix bump last updated time (#4971)
  Make `tracing::log` work in the runtime (#4863)
  sp-core: Improve docs generated by `generate_feature_enabled_macro` (#4968)
  [Backport] Version bumps  and  prdocs reordering from 1.14.0 (#4955)
  Assets: can_decrease/increase for destroying asset is not successful (#3286)
  ...
ordian added a commit that referenced this pull request Jul 18, 2024
* master: (125 commits)
  add elastic scaling MVP guide (#4663)
  Send PeerViewChange with high priority (#4755)
  [ci] Update forklift in CI image (#5032)
  Adjust base value for statement-distribution regression tests (#5028)
  [pallet_contracts] Add support for transient storage in contracts host functions (#4566)
  [1 / 5] Optimize logic for gossiping assignments (#4848)
  Remove `pallet-getter` usage from pallet-session (#4972)
  command-action: added scoped permissions to the github tokens (#5016)
  net/litep2p: Propagate ValuePut events to the network backend (#5018)
  rpc: add back rpc logger (#4952)
  Updated substrate-relay version for tests (#5017)
  Remove most all usage of `sp-std` (#5010)
  Use sp_runtime::traits::BadOrigin (#5011)
  network/tx: Ban peers with tx that fail to decode (#5002)
  Try State Hook for Bounties (#4563)
  [statement-distribution] Add metrics for distributed statements in V2 (#4554)
  added sync command (#4818)
  Bridges V2 refactoring backport and `pallet_bridge_messages` simplifications (#4935)
  xcm-executor: Improve logging (#4996)
  Remove usage of `sp-std` on templates (#5001)
  ...
TarekkMA pushed a commit to moonbeam-foundation/polkadot-sdk that referenced this pull request Aug 2, 2024
…cations (paritytech#4935)

## Summary

This PR contains migrated code from the Bridges V2
[branch](paritytech#4427) from the
old `parity-bridges-common`
[repo](https://github.com/paritytech/parity-bridges-common/tree/bridges-v2).
Even though the PR looks large, it does not (or should not) contain any
significant changes (also not relevant for audit).
This PR is a requirement for permissionless lanes, as they were
implemented on top of these changes.

## TODO

- [x] generate fresh weights for BridgeHubs
- [x] run `polkadot-fellows` bridges zombienet tests with actual runtime
1.2.5. or 1.2.6 to check compatibility
- ☑️ working, checked with 1.2.8 fellows BridgeHubs
- [x] run `polkadot-sdk` bridges zombienet tests
  - ☑️ with old relayer in CI (1.6.5) 
- [x] run `polkadot-sdk` bridges zombienet tests (locally) - with the
relayer based on this branch -
paritytech/parity-bridges-common#3022
- [x] check/fix relayer companion in bridges repo -
paritytech/parity-bridges-common#3022
- [x] extract pruning stuff to separate PR
paritytech#4944

Relates to:
paritytech/parity-bridges-common#2976
Relates to:
paritytech/parity-bridges-common#2451

---------

Signed-off-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Serban Iorga <serban@parity.io>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: command-bot <>
ordian added a commit that referenced this pull request Aug 6, 2024
* master: (130 commits)
  add elastic scaling MVP guide (#4663)
  Send PeerViewChange with high priority (#4755)
  [ci] Update forklift in CI image (#5032)
  Adjust base value for statement-distribution regression tests (#5028)
  [pallet_contracts] Add support for transient storage in contracts host functions (#4566)
  [1 / 5] Optimize logic for gossiping assignments (#4848)
  Remove `pallet-getter` usage from pallet-session (#4972)
  command-action: added scoped permissions to the github tokens (#5016)
  net/litep2p: Propagate ValuePut events to the network backend (#5018)
  rpc: add back rpc logger (#4952)
  Updated substrate-relay version for tests (#5017)
  Remove most all usage of `sp-std` (#5010)
  Use sp_runtime::traits::BadOrigin (#5011)
  network/tx: Ban peers with tx that fail to decode (#5002)
  Try State Hook for Bounties (#4563)
  [statement-distribution] Add metrics for distributed statements in V2 (#4554)
  added sync command (#4818)
  Bridges V2 refactoring backport and `pallet_bridge_messages` simplifications (#4935)
  xcm-executor: Improve logging (#4996)
  Remove usage of `sp-std` on templates (#5001)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T15-bridges This PR/Issue is related to bridges.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

5 participants