chore: upgrade to polkadot v1.11.0 #666

Roznovjak · 2024-08-12T11:34:43Z

NOTES:
All changes can be found here:

Breaking changes:

Remove prometheus RPC metrics substrate_rpc_requests_started and substrate_rpc_requests_finished

Notable changes

System Callbacks: The system pallet gets five new config items - all of which can be safely set to () as default.
- SingleBlockMigrations: replaces the Executive now for configuring migrations.
- MultiBlockMigrator: the pallet-migrations would be set here, if deployed.
- PreInherents: a hook that runs before any inherent.
- PostInherents: a hook to run between inherents and poll/MBM logic.
- PostTransactions: a hook to run after all transactions but before on_idle.
Add the ability for MessageQueue to process enqueued messages on idle
Rate limiting for RPC server which can be utilized by the CLI --rpc-rate-limit
Adds pallet-parameters that allows to have parameters for pallet configs that dynamically change at runtime
pallet-migrations is introduced that can be configured in the System of a runtime to act as multi-block migrator.
Add claim_assets extrinsic to pallet-xcm
The construct_runtime macro now generates a test to assert that all GenesisConfigs of all pallets can be build within the runtime.
XCM fee payment API: A runtime API was added for estimating the fees required for XCM execution and delivery.
- fn query_acceptable_payment_assets(xcm_version: Version) -> Result<Vec, Error>;
- fn query_xcm_weight(message: VersionedXcm) -> Result<Weight, Error>;
- fn query_weight_to_asset_fee(weight: Weight, asset: VersionedAssetId) -> Result<u128, Error>;
- fn query_delivery_fees(destination: VersionedLocation, message: VersionedXcm<()>) -> Result<VersionedAssets, Error>;
pallet-xcm's extrinsics teleport_assets and reserve_transfer_assets have been marked as deprecated. Please change their usage to the limited_teleport_assets and limited_reserve_transfer_assets, respectively; or use the generic/flexible transfer_assets extrinsic.
pallet-xcm's extrinsics execute and send have been marked as deprecated. Please change their usage to the new execute_blob and send_blob. There's a new utility in PolkadotJS Apps for encoding XCMs you can use: https://polkadot.js.org/apps/#/utilities/xcm
Migrations: prevent accidentally using inner unversioned migration instead of VersionedMigration
Adds optional HRMP notification handlers to the xcm-executor. These handlers are 3 new config types on the xcm-executor Config trait:
- HrmpNewChannelOpenRequestHandler
- HrmpChannelAcceptedHandler
- HrmpChannelClosingHandler
The runtime now can provide a number of predefined presets of RuntimeGenesisConfig struct. This presets are intended to be used in different deployments, e.g.: local, staging, etc, and should be included into the corresponding chain-specs. Having GenesisConfig presets in runtime allows to fully decouple node from runtime types (the problem is described in #1984).
- The GenesisBuilder API was adjusted to enable this functionality,
- (Breaking change) Old create_default_config method was removed, build_config was renamed to build_state. As a consequence a node won't be able to interact with genesis config for older runtimes. The cleanup was made for sake of API simplicity.
New call hrmp.establish_channel_with_system to allow parachains to establish a channel with a system parachain
(Breaking change) For all authority nodes, the node binary now enforces the presence of a network key, instead of auto-generating when it is absent.
pallet-xcm has a new extrinsic transfer_assets_using_type_and_then for transferring assets from local chain to destination chain using an explicit XCM transfer types for transferring the assets and the fees:
- TransferType::LocalReserve: transfer assets to sovereign account of destination chain and forward a notification XCM to dest to mint and deposit reserve-based assets to beneficiary.
- TransferType::DestinationReserve: burn local assets and forward a notification to dest chain to withdraw the reserve assets from this chain's sovereign account and deposit them to beneficiary.
- TransferType::RemoteReserve(reserve): burn local assets, forward XCM to reserve chain to move reserves from this chain's SA to dest chain's SA, and forward another XCM to dest to mint and deposit reserve-based assets to beneficiary. Typically the remote reserve is Asset Hub.
- TransferType::Teleport: burn local assets and forward XCM to dest chain to mint/teleport assets and deposit them to beneficiary. By default, an asset's reserve is its origin chain. But sometimes we may want to explicitly use another chain as reserve (as long as allowed by runtime IsReserve filter). This is very helpful for transferring assets with multiple configured reserves (such as Asset Hub ForeignAssets), when the transfer strictly depends on the used reserve location.
  E.g. For transferring a bridged Foreign Assets between local parachains, Asset Hub or the parachain that bridged the asset over must be used as the reserve location. Same when transferring bridged assets back across the bridge, the local bridging parachain must be used as the explicit reserve location.
  The new method takes a custom_xcm_on_dest parameter allowing the caller to specify what should happen to the transferred assets once they reach the dest chain. The custom_xcm_on_dest parameter should contains the instructions to execute on dest as a final step. Usually as simple as:  Xcm(vec![DepositAsset { assets: Wild(AllCounted(assets.len())), beneficiary }]),  but could be something more exotic like sending the assets even further.
Remove XCM SafeCallFilter for chains using Weights::v3
- SafeCallFilter was removed from Rococo and Westend relay and system chains as they all now use Weights::v3 which already accounts for call PoV size. This effectively removes artificial limitations on what users can XCM::Transact on these chains (blockspace limitations are still upheld).

github-actions · 2024-08-12T11:38:04Z

Crate versions that have been updated:

runtime-integration-tests: v1.0.2 -> v1.0.3
basilisk: v14.0.0 -> v15.0.0
pallet-asset-registry: v2.3.3 -> v2.3.4
pallet-marketplace: v5.0.18 -> v5.0.19
pallet-xyk-liquidity-mining: v1.1.11 -> v1.1.12
pallet-xyk-liquidity-mining-benchmarking: v1.0.17 -> v1.0.18
primitives: v6.6.2 -> v6.6.3
basilisk-runtime: v118.0.0 -> v119.0.0

Runtime version has been increased.

…_present integration test

Cargo.toml

primitives/Cargo.toml

runtime/basilisk/src/apis.rs

runtime/basilisk/src/assets.rs

dmoka · 2024-09-04T13:43:56Z

runtime/basilisk/src/xcm.rs

@@ -322,6 +325,7 @@ impl pallet_message_queue::Config for Runtime {
 	type HeapSize = MessageQueueHeapSize;
 	type MaxStale = MessageQueueMaxStale;
 	type ServiceWeight = MessageQueueServiceWeight;
+	type IdleMaxServiceWeight = ();


seems like an important field we should configure?! So the max service weight we want to use on idle.

default is none

but mby we could configure it to ServiceWeight so that we are able to process messages on idle as well

dmoka · 2024-09-05T07:05:05Z

runtime/basilisk/src/weights/pallet_xcm.rs

 //! WORST CASE MAP SIZE: `1000000`
 //! HOSTNAME: `bench-bot`, CPU: `Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz`
-//! WASM-EXECUTION: `Compiled`, CHAIN: `None`, DB CACHE: `1024`
+//! WASM-EXECUTION: `Compiled`, CHAIN: `Some("dev")`, DB CACHE: `1024`


is there any reason that sometimes we use DEV as chain, but sometimes we use None like here runtime/basilisk/src/weights/pallet_whitelist.rs

The reason is that some pallets can't be benchmarked with the old benchmarking command (there is a bug, the command line parameter --heap-pages is broken) so some pallets are benchmarked with new benchmarking tool. But this new tool uses a different chain spec and doesn't work with all pallets. As a result, we need to use 2 different benchmarking tools.

actually no. It looks like a different chain spec is used. But it should not be a problem.

mrq1911 · 2024-09-12T13:14:31Z

Remove prometheus RPC metrics substrate_rpc_requests_started and substrate_rpc_requests_finished

damn, i was using that... ty for letting me know @Roznovjak

mrq1911 · 2024-09-12T13:20:05Z

(Breaking change) For all authority nodes, the node binary now enforces the presence of a network key, instead of auto-generating when it is absent.

important breking change that will affect collators

mrq1911 · 2024-09-12T13:22:13Z

integration-tests/src/cross_chain_transfer.rs

 		));
 	});
 	Rococo::execute_with(|| {
-		assert_ok!(rococo_runtime::XcmPallet::reserve_transfer_assets(
+		assert_ok!(rococo_runtime::XcmPallet::limited_reserve_transfer_assets(


reserve_transfer_asset was removed?

deprecated, so I changed the test to get rid of the warning.

runtime/basilisk/src/apis.rs

mrq1911 · 2024-09-12T13:44:50Z

runtime/basilisk/src/weights/pallet_state_trie_migration.rs

@@ -18,30 +18,28 @@



do we really still need this pallet?

mrq1911 · 2024-09-12T13:48:34Z

runtime/basilisk/src/xcm.rs

@@ -322,6 +325,7 @@ impl pallet_message_queue::Config for Runtime {
 	type HeapSize = MessageQueueHeapSize;
 	type MaxStale = MessageQueueMaxStale;
 	type ServiceWeight = MessageQueueServiceWeight;
+	type IdleMaxServiceWeight = ();


default is none

mrq1911 · 2024-09-12T13:49:24Z

runtime/basilisk/src/xcm.rs

@@ -322,6 +325,7 @@ impl pallet_message_queue::Config for Runtime {
 	type HeapSize = MessageQueueHeapSize;
 	type MaxStale = MessageQueueMaxStale;
 	type ServiceWeight = MessageQueueServiceWeight;
+	type IdleMaxServiceWeight = ();


but mby we could configure it to ServiceWeight so that we are able to process messages on idle as well

mrq1911 · 2024-09-12T13:49:51Z

scripts/benchmarking.sh

I would keep it. Let's hope that the issue with heap-pages will be fixed in the next release and we can use it again. There is also a standalone tool for benchmarking, omni-bencher, which can also be used to benchmark all pallets, but it uses default (empty) chain spec and doesn't work with all our pallets.

Roznovjak added 2 commits August 12, 2024 13:33

fix build

c10c5a8

formatting

b535204

Roznovjak self-assigned this Aug 12, 2024

Roznovjak added 26 commits August 14, 2024 13:14

fix some integration tests

517c000

add migration

5948676

attempt to fix failing integration test

61644b3

fix basilisk_should_swap_assets_coming_from_karura_when_onchain_route…

9a0fe12

…_present integration test

formatting

cab3512

fix integration and benchmarking test

1f53f91

satisfy clippy

8176296

formatting

f44d42a

remove unused dependency

5982df4

remove fix in cargo.toml that is no longer needed

017088b

add GenesisBuilder runtime API

670c098

formatting

a4cd52b

bump crate versions

7ca93bd

move runtime apis to separate file

fafc2cf

fix dependencies

85c1f69

update pallet_xcm benchmarking helper functions

8c77833

fix pallet_xcm benchmarks

668380c

rebenchmarking

5c2e1ad

fix router benchmarks

c1ec167

rebenchmark using ref machine

4c827b0

rebenchmark router

4b0fb4d

fix clippy

b52420b

fix integration test

bfb80a8

formatting

b5ff2c4

rebenchmark with correct params

e63bfd7

rebenchmark router

a22e058

Roznovjak added 2 commits August 29, 2024 12:02

fix integration tests

eaf9496

formatting

fdeca78

Roznovjak marked this pull request as ready for review August 29, 2024 11:11

Roznovjak requested review from mrq1911, jak-pan, dmoka and enthusiastmartin August 29, 2024 11:11

Roznovjak added 7 commits August 29, 2024 14:13

rebenchmark collator selection pallet with old benchmarking command

be8d34c

add storage proof size host functions

7cd3f27

use unified host function

a7fb136

remove old params for previous migrations

24c5b57

enable proof recording during block import

81917e5

update name of polkadot-sdk repo in zombienet configs

4c73aa6

formatting

52c1195

dmoka reviewed Sep 5, 2024

View reviewed changes

Roznovjak added 7 commits September 5, 2024 13:10

implement XcmPaymentApi

fbac6b0

use workspace dependencies

29577e1

Merge branch 'master' into polkadot-v1.11.0

ceaecba

remove duplicate dependency

ba21785

formatting

e4c8c2d

bump crate versions

4207ac3

fix name of the polkadot-sdk repo in zombienet config

f3397b8

fix error from router benchmarking test

44d32ce

mrq1911 approved these changes Sep 12, 2024

View reviewed changes

mrq1911 and others added 2 commits September 12, 2024 15:57

Merge branch 'master' into polkadot-v1.11.0

8976276

set IdleMaxServiceWeight

34a5bdd

mrq1911 merged commit d6f885a into master Sep 13, 2024
7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: upgrade to polkadot v1.11.0 #666

chore: upgrade to polkadot v1.11.0 #666

Roznovjak commented Aug 12, 2024 •

edited

Loading

github-actions bot commented Aug 12, 2024 •

edited

Loading

dmoka Sep 4, 2024

mrq1911 Sep 12, 2024

mrq1911 Sep 12, 2024

dmoka Sep 5, 2024

Roznovjak Sep 10, 2024

Roznovjak Sep 11, 2024

mrq1911 commented Sep 12, 2024

mrq1911 commented Sep 12, 2024 •

edited

Loading

mrq1911 Sep 12, 2024

Roznovjak Sep 12, 2024

mrq1911 Sep 12, 2024

mrq1911 Sep 12, 2024

mrq1911 Sep 12, 2024

mrq1911 Sep 12, 2024

Roznovjak Sep 12, 2024

chore: upgrade to polkadot v1.11.0 #666

chore: upgrade to polkadot v1.11.0 #666

Conversation

Roznovjak commented Aug 12, 2024 • edited Loading

github-actions bot commented Aug 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrq1911 commented Sep 12, 2024

mrq1911 commented Sep 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Roznovjak commented Aug 12, 2024 •

edited

Loading

github-actions bot commented Aug 12, 2024 •

edited

Loading

mrq1911 commented Sep 12, 2024 •

edited

Loading