From 1c1cd40e3a2c8129561cbc78a265f7a541da7545 Mon Sep 17 00:00:00 2001 From: Fedor Sakharov Date: Wed, 17 Jun 2020 19:50:39 +0300 Subject: [PATCH 01/11] Router module initial commit --- roadmap/implementors-guide/src/SUMMARY.md | 1 + roadmap/implementors-guide/src/runtime/inclusion.md | 2 ++ roadmap/implementors-guide/src/type-definitions.md | 6 ++++++ 3 files changed, 9 insertions(+) diff --git a/roadmap/implementors-guide/src/SUMMARY.md b/roadmap/implementors-guide/src/SUMMARY.md index 17b370385366..94d1257dcdac 100644 --- a/roadmap/implementors-guide/src/SUMMARY.md +++ b/roadmap/implementors-guide/src/SUMMARY.md @@ -13,6 +13,7 @@ - [Inclusion Module](runtime/inclusion.md) - [InclusionInherent Module](runtime/inclusioninherent.md) - [Validity Module](runtime/validity.md) + - [Router Module](runtime/router.md) - [Node Architecture](node/README.md) - [Subsystems and Jobs](node/subsystems-and-jobs.md) - [Overseer](node/overseer.md) diff --git a/roadmap/implementors-guide/src/runtime/inclusion.md b/roadmap/implementors-guide/src/runtime/inclusion.md index 944cd1def2f2..61d91c462211 100644 --- a/roadmap/implementors-guide/src/runtime/inclusion.md +++ b/roadmap/implementors-guide/src/runtime/inclusion.md @@ -54,7 +54,9 @@ All failed checks should lead to an unrecoverable error making the block invalid 1. check that each candidate corresponds to a scheduled core and that they are ordered in ascending order by `ParaId`. 1. Ensure that any code upgrade scheduled by the candidate does not happen within `config.validation_upgrade_frequency` of the currently scheduled upgrade, if any, comparing against the value of `Paras::FutureCodeUpgrades` for the given para ID. 1. check the backing of the candidate using the signatures and the bitfields. + 1. check that the upwards messages are not exceeding `config.max_upwards_queue_count` and `config.watermark_queue_size` parameters. 1. create an entry in the `PendingAvailability` map for each backed candidate with a blank `availability_votes` bitfield. + 1. call `Router::queue_upward_messages` for each backed candidate. 1. Return a `Vec` of all scheduled cores of the list of passed assignments that a candidate was successfully backed for, sorted ascending by CoreIndex. * `enact_candidate(relay_parent_number: BlockNumber, AbridgedCandidateReceipt)`: 1. If the receipt contains a code upgrade, Call `Paras::schedule_code_upgrade(para_id, code, relay_parent_number + config.validationl_upgrade_delay)`. diff --git a/roadmap/implementors-guide/src/type-definitions.md b/roadmap/implementors-guide/src/type-definitions.md index d8c90bba50d0..2b5677137d2a 100644 --- a/roadmap/implementors-guide/src/type-definitions.md +++ b/roadmap/implementors-guide/src/type-definitions.md @@ -232,6 +232,12 @@ struct HostConfiguration { pub thread_availability_period: BlockNumber, /// The amount of blocks ahead to schedule parathreads. pub scheduling_lookahead: u32, + /// Total number of individual messages allowed in the parachain -> relay-chain message queue. + pub max_upwards_queue_count: u32, + /// Total size of messages allowed in the parachain -> relay-chain message queue before which + /// no further messages may be added to it. If it exceeds this then the queue may contain only + /// a single message. + pub watermark_queue_size: u32, } ``` From 46fa5d14c5f7d8c7ded9c58ad6c868110e312bbc Mon Sep 17 00:00:00 2001 From: Fedor Sakharov Date: Wed, 17 Jun 2020 19:53:20 +0300 Subject: [PATCH 02/11] Add the router.md itself --- .../implementors-guide/src/runtime/router.md | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 roadmap/implementors-guide/src/runtime/router.md diff --git a/roadmap/implementors-guide/src/runtime/router.md b/roadmap/implementors-guide/src/runtime/router.md new file mode 100644 index 000000000000..9442f576130c --- /dev/null +++ b/roadmap/implementors-guide/src/runtime/router.md @@ -0,0 +1,28 @@ +# Router Module + +The Router module is responsible for storing and dispatching Upwards and Downwards messages from and to parachains respectively. It is intended to later handle the XCMP logic as well. + +## Storage + +Storage layout: + +```rust + +/// Messages ready to be dispatched onto the relay chain. +/// This is subject to `max_upwards_queue_count` and +///`watermark_queue_size` from `HostConfiguration`. +RelayDispatchQueues: map ParaId => Vec; +/// Size of the dispatch queues. Caches sizes of the queues in `RelayDispatchQueue`. +/// First item in the tuple is the count of messages and second +/// is the total length (in bytes) of the message payloads. +RelayDispatchQueueSize: map ParaId => (u32, u32); +/// The ordered list of `ParaId`s that have a `RelayDispatchQueue` entry. +NeedsDispatch: Vec; +``` + +## Routines + +* `queue_upward_messages(AttestedCandidate)`: + 1. Updates `NeedsDispatch`, and enqueues upward messages into `RelayDispatchQueue` and modifies the respective entry in `RelayDispatchQueueSize`. +* `dispatch_upward_messages(ParaId)`: + 1. If `NeedsDispatch` contains an entry passed as an input parameter start dispatching messages from it's respective entry in `RelayDispatchQueues`. The dispatch is done in the FIFO order and it drains the queue and removes it from `RelayDispatchQueues`. From c431b457edfd84ab650b4d7a41c0ca6baa693102 Mon Sep 17 00:00:00 2001 From: Fedor Sakharov Date: Wed, 17 Jun 2020 20:46:38 +0300 Subject: [PATCH 03/11] Enqueue only on enact_candidate --- roadmap/implementors-guide/src/runtime/inclusion.md | 2 +- roadmap/implementors-guide/src/runtime/router.md | 9 +++++++-- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/roadmap/implementors-guide/src/runtime/inclusion.md b/roadmap/implementors-guide/src/runtime/inclusion.md index 61d91c462211..0e57a820adcc 100644 --- a/roadmap/implementors-guide/src/runtime/inclusion.md +++ b/roadmap/implementors-guide/src/runtime/inclusion.md @@ -56,11 +56,11 @@ All failed checks should lead to an unrecoverable error making the block invalid 1. check the backing of the candidate using the signatures and the bitfields. 1. check that the upwards messages are not exceeding `config.max_upwards_queue_count` and `config.watermark_queue_size` parameters. 1. create an entry in the `PendingAvailability` map for each backed candidate with a blank `availability_votes` bitfield. - 1. call `Router::queue_upward_messages` for each backed candidate. 1. Return a `Vec` of all scheduled cores of the list of passed assignments that a candidate was successfully backed for, sorted ascending by CoreIndex. * `enact_candidate(relay_parent_number: BlockNumber, AbridgedCandidateReceipt)`: 1. If the receipt contains a code upgrade, Call `Paras::schedule_code_upgrade(para_id, code, relay_parent_number + config.validationl_upgrade_delay)`. > TODO: Note that this is safe as long as we never enact candidates where the relay parent is across a session boundary. In that case, which we should be careful to avoid with contextual execution, the configuration might have changed and the para may de-sync from the host's understanding of it. + 1. call `Router::queue_upward_messages` for each backed candidate. 1. Call `Paras::note_new_head` using the `HeadData` from the receipt and `relay_parent_number`. * `collect_pending`: diff --git a/roadmap/implementors-guide/src/runtime/router.md b/roadmap/implementors-guide/src/runtime/router.md index 9442f576130c..bbfa330bf53d 100644 --- a/roadmap/implementors-guide/src/runtime/router.md +++ b/roadmap/implementors-guide/src/runtime/router.md @@ -22,7 +22,12 @@ NeedsDispatch: Vec; ## Routines -* `queue_upward_messages(AttestedCandidate)`: +* `queue_upward_messages(AbridgedCandidateReceipt)`: 1. Updates `NeedsDispatch`, and enqueues upward messages into `RelayDispatchQueue` and modifies the respective entry in `RelayDispatchQueueSize`. -* `dispatch_upward_messages(ParaId)`: +* `dispatch_upward_messages()`: 1. If `NeedsDispatch` contains an entry passed as an input parameter start dispatching messages from it's respective entry in `RelayDispatchQueues`. The dispatch is done in the FIFO order and it drains the queue and removes it from `RelayDispatchQueues`. + +## Initialization + + > TODO: On initalization or finalization or when exactly? + 1. Dispatch queued upward messages using `dispatch_upward_messages`. From b21b6dbecb3cae2fc4017d13c81ad5c37217c471 Mon Sep 17 00:00:00 2001 From: Fedor Sakharov Date: Wed, 17 Jun 2020 22:04:45 +0300 Subject: [PATCH 04/11] Initialization concerns --- roadmap/implementors-guide/src/runtime/router.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/roadmap/implementors-guide/src/runtime/router.md b/roadmap/implementors-guide/src/runtime/router.md index bbfa330bf53d..60260916cc49 100644 --- a/roadmap/implementors-guide/src/runtime/router.md +++ b/roadmap/implementors-guide/src/runtime/router.md @@ -2,12 +2,17 @@ The Router module is responsible for storing and dispatching Upwards and Downwards messages from and to parachains respectively. It is intended to later handle the XCMP logic as well. +For each enacted block the `enqueue_upward_messages` entry-point is called. All dispatching logic is done by the `dispatch_upward_messages` entry-point. This entry-point is mandatory, in that it must be envoked once within every block. + +This module does not have the same initialization/finalization concerns as the others, it requires that its entry points be triggered + 1. When candidate is enacted. + 1. Upon new block initialization. + ## Storage Storage layout: ```rust - /// Messages ready to be dispatched onto the relay chain. /// This is subject to `max_upwards_queue_count` and ///`watermark_queue_size` from `HostConfiguration`. From 588d309ebd5c23ec286663b41cf2bc3078be8193 Mon Sep 17 00:00:00 2001 From: Fedor Sakharov Date: Wed, 17 Jun 2020 22:10:59 +0300 Subject: [PATCH 05/11] Ignore the snippet --- roadmap/implementors-guide/src/runtime/router.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roadmap/implementors-guide/src/runtime/router.md b/roadmap/implementors-guide/src/runtime/router.md index 60260916cc49..622b327a5fc2 100644 --- a/roadmap/implementors-guide/src/runtime/router.md +++ b/roadmap/implementors-guide/src/runtime/router.md @@ -12,7 +12,7 @@ This module does not have the same initialization/finalization concerns as the o Storage layout: -```rust +```rust,ignore /// Messages ready to be dispatched onto the relay chain. /// This is subject to `max_upwards_queue_count` and ///`watermark_queue_size` from `HostConfiguration`. From 88bf5f5f56b03d915b34b4686e0d514dd8445fb1 Mon Sep 17 00:00:00 2001 From: Fedor Sakharov Date: Thu, 18 Jun 2020 12:42:35 +0300 Subject: [PATCH 06/11] Update roadmap/implementors-guide/src/runtime/router.md Co-authored-by: Robert Habermeier --- roadmap/implementors-guide/src/runtime/router.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roadmap/implementors-guide/src/runtime/router.md b/roadmap/implementors-guide/src/runtime/router.md index 622b327a5fc2..259812ceacfb 100644 --- a/roadmap/implementors-guide/src/runtime/router.md +++ b/roadmap/implementors-guide/src/runtime/router.md @@ -2,7 +2,7 @@ The Router module is responsible for storing and dispatching Upwards and Downwards messages from and to parachains respectively. It is intended to later handle the XCMP logic as well. -For each enacted block the `enqueue_upward_messages` entry-point is called. All dispatching logic is done by the `dispatch_upward_messages` entry-point. This entry-point is mandatory, in that it must be envoked once within every block. +For each enacted block the `enqueue_upward_messages` entry-point is called. All dispatching logic is done by the `dispatch_upward_messages` entry-point. This entry-point is mandatory, in that it must be invoked once within every block. This module does not have the same initialization/finalization concerns as the others, it requires that its entry points be triggered 1. When candidate is enacted. From b29789fe74816f28415d14c53e65faf627e31096 Mon Sep 17 00:00:00 2001 From: Fedor Sakharov Date: Thu, 18 Jun 2020 16:48:27 +0300 Subject: [PATCH 07/11] Dispatch messages on finalization --- .../implementors-guide/src/runtime/router.md | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/roadmap/implementors-guide/src/runtime/router.md b/roadmap/implementors-guide/src/runtime/router.md index 259812ceacfb..5e40be549ef8 100644 --- a/roadmap/implementors-guide/src/runtime/router.md +++ b/roadmap/implementors-guide/src/runtime/router.md @@ -2,11 +2,7 @@ The Router module is responsible for storing and dispatching Upwards and Downwards messages from and to parachains respectively. It is intended to later handle the XCMP logic as well. -For each enacted block the `enqueue_upward_messages` entry-point is called. All dispatching logic is done by the `dispatch_upward_messages` entry-point. This entry-point is mandatory, in that it must be invoked once within every block. - -This module does not have the same initialization/finalization concerns as the others, it requires that its entry points be triggered - 1. When candidate is enacted. - 1. Upon new block initialization. +For each enacted block the `queue_upward_messages` entry-point is called. ## Storage @@ -25,14 +21,15 @@ RelayDispatchQueueSize: map ParaId => (u32, u32); NeedsDispatch: Vec; ``` +## Initialization + +No initialization routine runs for this module. + ## Routines * `queue_upward_messages(AbridgedCandidateReceipt)`: 1. Updates `NeedsDispatch`, and enqueues upward messages into `RelayDispatchQueue` and modifies the respective entry in `RelayDispatchQueueSize`. -* `dispatch_upward_messages()`: - 1. If `NeedsDispatch` contains an entry passed as an input parameter start dispatching messages from it's respective entry in `RelayDispatchQueues`. The dispatch is done in the FIFO order and it drains the queue and removes it from `RelayDispatchQueues`. -## Initialization +## Finalization - > TODO: On initalization or finalization or when exactly? - 1. Dispatch queued upward messages using `dispatch_upward_messages`. + 1. Dispatch queued upward messages from `RelayDispatchQueues` in a FIFO order applying the `config.watermark_queue_size` and `config.max_upwards_queue_count` limits. From 2460e53fd6060bf0db93219d5992e178d81f2a68 Mon Sep 17 00:00:00 2001 From: Fedor Sakharov Date: Thu, 18 Jun 2020 18:54:48 +0300 Subject: [PATCH 08/11] More fixes from review --- roadmap/implementors-guide/src/SUMMARY.md | 1 + roadmap/implementors-guide/src/runtime/inclusion.md | 2 +- roadmap/implementors-guide/src/runtime/router.md | 8 ++++---- roadmap/implementors-guide/src/types/runtime.md | 4 ++-- 4 files changed, 8 insertions(+), 7 deletions(-) diff --git a/roadmap/implementors-guide/src/SUMMARY.md b/roadmap/implementors-guide/src/SUMMARY.md index 6de3dcddf7ed..194fd1bde608 100644 --- a/roadmap/implementors-guide/src/SUMMARY.md +++ b/roadmap/implementors-guide/src/SUMMARY.md @@ -45,6 +45,7 @@ - [Overseer and Subsystem Protocol](types/overseer-protocol.md) - [Runtime](types/runtime.md) - [Chain](types/chain.md) + - [Messages](types/messages.md) [Glossary](glossary.md) [Further Reading](further-reading.md) diff --git a/roadmap/implementors-guide/src/runtime/inclusion.md b/roadmap/implementors-guide/src/runtime/inclusion.md index 0e57a820adcc..c5df672d8eaf 100644 --- a/roadmap/implementors-guide/src/runtime/inclusion.md +++ b/roadmap/implementors-guide/src/runtime/inclusion.md @@ -54,7 +54,7 @@ All failed checks should lead to an unrecoverable error making the block invalid 1. check that each candidate corresponds to a scheduled core and that they are ordered in ascending order by `ParaId`. 1. Ensure that any code upgrade scheduled by the candidate does not happen within `config.validation_upgrade_frequency` of the currently scheduled upgrade, if any, comparing against the value of `Paras::FutureCodeUpgrades` for the given para ID. 1. check the backing of the candidate using the signatures and the bitfields. - 1. check that the upwards messages are not exceeding `config.max_upwards_queue_count` and `config.watermark_queue_size` parameters. + 1. check that the upward messages are not exceeding `config.max_upward_queue_count` and `config.watermark_upward_queue_size` parameters. 1. create an entry in the `PendingAvailability` map for each backed candidate with a blank `availability_votes` bitfield. 1. Return a `Vec` of all scheduled cores of the list of passed assignments that a candidate was successfully backed for, sorted ascending by CoreIndex. * `enact_candidate(relay_parent_number: BlockNumber, AbridgedCandidateReceipt)`: diff --git a/roadmap/implementors-guide/src/runtime/router.md b/roadmap/implementors-guide/src/runtime/router.md index 5e40be549ef8..fff8e78920f7 100644 --- a/roadmap/implementors-guide/src/runtime/router.md +++ b/roadmap/implementors-guide/src/runtime/router.md @@ -1,6 +1,6 @@ # Router Module -The Router module is responsible for storing and dispatching Upwards and Downwards messages from and to parachains respectively. It is intended to later handle the XCMP logic as well. +The Router module is responsible for storing and dispatching Upward and Downward messages from and to parachains respectively. It is intended to later handle the XCMP logic as well. For each enacted block the `queue_upward_messages` entry-point is called. @@ -10,8 +10,8 @@ Storage layout: ```rust,ignore /// Messages ready to be dispatched onto the relay chain. -/// This is subject to `max_upwards_queue_count` and -///`watermark_queue_size` from `HostConfiguration`. +/// This is subject to `max_upward_queue_count` and +/// `watermark_queue_size` from `HostConfiguration`. RelayDispatchQueues: map ParaId => Vec; /// Size of the dispatch queues. Caches sizes of the queues in `RelayDispatchQueue`. /// First item in the tuple is the count of messages and second @@ -32,4 +32,4 @@ No initialization routine runs for this module. ## Finalization - 1. Dispatch queued upward messages from `RelayDispatchQueues` in a FIFO order applying the `config.watermark_queue_size` and `config.max_upwards_queue_count` limits. + 1. Dispatch queued upward messages from `RelayDispatchQueues` in a FIFO order applying the `config.watermark_upward_queue_size` and `config.max_upward_queue_count` limits. diff --git a/roadmap/implementors-guide/src/types/runtime.md b/roadmap/implementors-guide/src/types/runtime.md index 6ee6a1fa605c..ebdbfe15c502 100644 --- a/roadmap/implementors-guide/src/types/runtime.md +++ b/roadmap/implementors-guide/src/types/runtime.md @@ -35,10 +35,10 @@ struct HostConfiguration { /// The amount of blocks ahead to schedule parathreads. pub scheduling_lookahead: u32, /// Total number of individual messages allowed in the parachain -> relay-chain message queue. - pub max_upwards_queue_count: u32, + pub max_upward_queue_count: u32, /// Total size of messages allowed in the parachain -> relay-chain message queue before which /// no further messages may be added to it. If it exceeds this then the queue may contain only /// a single message. - pub watermark_queue_size: u32, + pub watermark_upward_queue_size: u32, } ``` From 8d750f20a820332afef256ef715520d48e876783 Mon Sep 17 00:00:00 2001 From: Fedor Sakharov Date: Thu, 18 Jun 2020 19:08:24 +0300 Subject: [PATCH 09/11] Update roadmap/implementors-guide/src/SUMMARY.md Co-authored-by: Robert Habermeier --- roadmap/implementors-guide/src/SUMMARY.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roadmap/implementors-guide/src/SUMMARY.md b/roadmap/implementors-guide/src/SUMMARY.md index 194fd1bde608..b5dc3840c605 100644 --- a/roadmap/implementors-guide/src/SUMMARY.md +++ b/roadmap/implementors-guide/src/SUMMARY.md @@ -45,7 +45,7 @@ - [Overseer and Subsystem Protocol](types/overseer-protocol.md) - [Runtime](types/runtime.md) - [Chain](types/chain.md) - - [Messages](types/messages.md) + - [Messages](types/messages.md) [Glossary](glossary.md) [Further Reading](further-reading.md) From 861010e31f1710af38a4237a6abcd93e979ca2a2 Mon Sep 17 00:00:00 2001 From: Fedor Sakharov Date: Thu, 18 Jun 2020 19:21:14 +0300 Subject: [PATCH 10/11] Adds router to initializer order --- roadmap/implementors-guide/src/runtime/initializer.md | 1 + 1 file changed, 1 insertion(+) diff --git a/roadmap/implementors-guide/src/runtime/initializer.md b/roadmap/implementors-guide/src/runtime/initializer.md index 1d343d7a4087..e184f0d829a6 100644 --- a/roadmap/implementors-guide/src/runtime/initializer.md +++ b/roadmap/implementors-guide/src/runtime/initializer.md @@ -17,6 +17,7 @@ The other modules are initialized in this order: 1. Scheduler 1. Inclusion 1. Validity. +1. Router. The [Configuration Module](configuration.html) is first, since all other modules need to operate under the same configuration as each other. It would lead to inconsistency if, for example, the scheduler ran first and then the configuration was updated before the Inclusion module. From 3cf5e8f0aaf60e2b9b31b700bceea291522c0d48 Mon Sep 17 00:00:00 2001 From: Fedor Sakharov Date: Thu, 18 Jun 2020 21:49:49 +0300 Subject: [PATCH 11/11] Adds messages.md --- .../implementors-guide/src/types/messages.md | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 roadmap/implementors-guide/src/types/messages.md diff --git a/roadmap/implementors-guide/src/types/messages.md b/roadmap/implementors-guide/src/types/messages.md new file mode 100644 index 000000000000..14218eb0330f --- /dev/null +++ b/roadmap/implementors-guide/src/types/messages.md @@ -0,0 +1,28 @@ +# Message types + +Types of messages that are passed between parachains and the relay chain: UMP, DMP, XCMP. + +## Upward Message + +A type of messages dispatched from a parachain to the relay chain. + +```rust,ignore +enum ParachainDispatchOrigin { + /// As a simple `Origin::Signed`, using `ParaId::account_id` as its value. This is good when + /// interacting with standard modules such as `balances`. + Signed, + /// As the special `Origin::Parachain(ParaId)`. This is good when interacting with parachain- + /// aware modules which need to succinctly verify that the origin is a parachain. + Parachain, + /// As the simple, superuser `Origin::Root`. This can only be done on specially permissioned + /// parachains. + Root, +} + +struct UpwardMessage { + /// The origin for the message to be sent from. + pub origin: ParachainDispatchOrigin, + /// The message data. + pub data: Vec, +} +```