FRAME: Create TransactionExtension as a replacement for SignedExtension

[gavofyork]

Closes #2160

First part of Extrinsic Horizon

Introduces a new trait TransactionExtension to replace SignedExtension. Introduce the idea of transactions which obey the runtime's extensions and have according Extension data (né Extra data) yet do not have hard-coded signatures.

Deprecate the terminology of "Unsigned" when used for transactions/extrinsics owing to there now being "proper" unsigned transactions which obey the extension framework and "old-style" unsigned which do not. Instead we have General for the former and Bare for the latter. (Ultimately, the latter will be phased out as a type of transaction, and Bare will only be used for Inherents.)

Types of extrinsic are now therefore:

• Bare (no hardcoded signature, no Extra data; used to be known as "Unsigned")
  • Bare transactions (deprecated): Gossiped, validated with ValidateUnsigned (deprecated) and the _bare_compat bits of TransactionExtension (deprecated).
  • Inherents: Not gossiped, validated with ProvideInherent.
• Extended (Extra data): Gossiped, validated via TransactionExtension.
  • Signed transactions (with a hardcoded signature).
  • General transactions (without a hardcoded signature).

TransactionExtension differs from SignedExtension because:

• A signature on the underlying transaction may validly not be present.
• It may alter the origin during validation.
• pre_dispatch is renamed to prepare and need not contain the checks present in validate.
• validate and prepare is passed an Origin rather than a AccountId.
• validate may pass arbitrary information into prepare via a new user-specifiable type Val.
• AdditionalSigned/additional_signed is renamed to Implicit/implicit. It is encoded for the entire transaction and passed in to each extension as a new argument to validate. This facilitates the ability of extensions to acts as underlying crypto.

There is a new DispatchTransaction trait which contains only default function impls and is impl'ed for any TransactionExtension impler. It provides several utility functions which reduce some of the tedium from using TransactionExtension (indeed, none of its regular functions should now need to be called directly).

Three transaction version discriminator ("versions") are now permissible:

• 0b000000100: Bare (used to be called "Unsigned"): contains Signature or Extra (extension data). After bare transactions are no longer supported, this will strictly identify an Inherents only.
• 0b100000100: Old-school "Signed" Transaction: contains Signature and Extra (extension data).
• 0b010000100: New-school "General" Transaction: contains Extra (extension data), but no Signature.

For the New-school General Transaction, it becomes trivial for authors to publish extensions to the mechanism for authorizing an Origin, e.g. through new kinds of key-signing schemes, ZK proofs, pallet state, mutations over pre-authenticated origins or any combination of the above.

Code Migration

NOW: Getting it to build

Wrap your SignedExtensions in AsTransactionExtension. This should be accompanied by renaming your aggregate type in line with the new terminology. E.g. Before:

/// The SignedExtension to the basic transaction logic.
pub type SignedExtra = (
	/* snip */
	MySpecialSignedExtension,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>;

After:

/// The extension to the basic transaction logic.
pub type TxExtension = (
	/* snip */
	AsTransactionExtension<MySpecialSignedExtension>,
);
/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
	generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, TxExtension>;

You'll also need to alter any transaction building logic to add a .into() to make the conversion happen. E.g. Before:

fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let extra: SignedExtra = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */),
	);
	let payload = SignedPayload::new(call.clone(), extra.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		extra,
	)
}

After:

fn construct_extrinsic(
		/* snip */
) -> UncheckedExtrinsic {
	let tx_ext: TxExtension = (
		/* snip */
		MySpecialSignedExtension::new(/* snip */).into(),
	);
	let payload = SignedPayload::new(call.clone(), tx_ext.clone()).unwrap();
	let signature = payload.using_encoded(|e| sender.sign(e));
	UncheckedExtrinsic::new_signed(
		/* snip */
		Signature::Sr25519(signature),
		tx_ext,
	)
}

SOON: Migrating to TransactionExtension

Most SignedExtensions can be trivially converted to become a TransactionExtension. There are a few things to know.

• Instead of a single trait like SignedExtension, you should now implement two traits individually: TransactionExtensionBase and TransactionExtension.
• Weights are now a thing and must be provided via the new function fn weight.

TransactionExtensionBase

This trait takes care of anything which is not dependent on types specific to your runtime, most notably Call.

• AdditionalSigned/additional_signed is renamed to Implicit/implicit.
• Weight must be returned by implementing the weight function. If your extension is associated with a pallet, you'll probably want to do this via the pallet's existing benchmarking infrastructure.

TransactionExtension

Generally:

• pre_dispatch is now prepare and you should not reexecute the validate functionality in there!
• You don't get an account ID any more; you get an origin instead. If you need to presume an account ID, then you can use the trait function AsSystemOriginSigner::as_system_origin_signer.
• You get an additional ticket, similar to Pre, called Val. This defines data which is passed from validate into prepare. This is important since you should not be duplicating logic from validate to prepare, you need a way of passing your working from the former into the latter. This is it.
• This trait takes two type parameters: Call and Context. Call is the runtime call type which used to be an associated type; you can just move it to become a type parameter for your trait impl. Context is not currently used and you can safely implement over it as an unbounded type.
• There's no AccountId associated type any more. Just remove it.

Regarding validate:

• You get three new parameters in validate; all can be ignored when migrating from SignedExtension.
• validate returns a tuple on success; the second item in the tuple is the new ticket type Self::Val which gets passed in to prepare. If you use any information extracted during validate (off-chain and on-chain, non-mutating) in prepare (on-chain, mutating) then you can pass it through with this. For the tuple's last item, just return the origin argument.

Regarding prepare:

• This is renamed from pre_dispatch, but there is one change:
• FUNCTIONALITY TO VALIDATE THE TRANSACTION NEED NOT BE DUPLICATED FROM validate!!
• (This is different to SignedExtension which was required to run the same checks in pre_dispatch as in validate.)

Regarding post_dispatch:

• Since there are no unsigned transactions handled by TransactionExtension, Pre is always defined, so the first parameter is Self::Pre rather than Option<Self::Pre>.

If you make use of SignedExtension::validate_unsigned or SignedExtension::pre_dispatch_unsigned, then:

• Just use the regular versions of these functions instead.
• Have your logic execute in the case that the origin is None.
• Ensure your transaction creation logic creates a General Transaction rather than a Bare Transaction; this means having to include all TransactionExtensions' data.
• ValidateUnsigned can still be used (for now) if you need to be able to construct transactions which contain none of the extension data, however these will be phased out in stage 2 of the Transactions Horizon, so you should consider moving to an extension-centric design.

TODO

• Introduce CheckSignature impl of TransactionExtension to ensure it's possible to have crypto be done wholly in a TransactionExtension.
• Deprecate SignedExtension and move all uses in codebase to TransactionExtension.
  • ChargeTransactionPayment
  • DummyExtension
  • ChargeAssetTxPayment (asset-tx-payment)
  • ChargeAssetTxPayment (asset-conversion-tx-payment)
  • CheckWeight
  • CheckTxVersion
  • CheckSpecVersion
  • CheckNonce
  • CheckNonZeroSender
  • CheckMortality
  • CheckGenesis
  • CheckOnlySudoAccount
  • WatchDummy
  • PrevalidateAttests
  • GenericSignedExtension
  • SignedExtension (chain-polkadot-bulletin)
  • RefundSignedExtensionAdapter
• Implement fn weight across the board.
• Go through all pre-existing extensions which assume an account signer and explicitly handle the possibility of another kind of origin.
  • CheckNonce should probably succeed in the case of a non-account origin.
  • CheckNonZeroSender should succeed in the case of a non-account origin.
  • ChargeTransactionPayment and family should fail in the case of a non-account origin.
  • [ ]
• Fix any broken tests.

[xlc]

Should we have a RFC to describe the extrinsic format change? Yeah I can read the diff but I prefer to read a spec in this case.

The API changes can be considered internal details and no need to be spec'ed but extrinsic format is part of external interface and should be discussed and documented.

[gavofyork]

Should we have a RFC to describe the extrinsic format change? Yeah I can read the diff but I prefer to read a spec in this case.

The API changes can be considered internal details and no need to be spec'ed but extrinsic format is part of external interface and should be discussed and documented.

This PR makes no alterations to the protocol of the Polkadot Relay or system chains as defined by the extrinsic datagrams which are acceptable or their meaning.

Furthermore, we do not (yet) have a convention of blanket requiring preapproval via an RFC on arbitrarily small changes which would alter transaction formats used by the Polkadot Relay and system chains. If you feel that such a move would be in the interests of Polkadot, I suggest you consider writing an RFC to propose it.

[bkchr]

Why do we need this? I couldn't find anything where this is really required to be separate? Asking because I think that having two traits makes this more complicated to handle, as you need to implement both traits always.