[Question]: TypeRegistry metadata

[TarikGul]

We have a library that constructs tx's "offline" (I put it in quotes because is it really ever offline? 🤷‍♂️ Not really)
I recently made an update where we construct the metadata, set it into the Registry, and set the the metadata used in the unsignedTx as registry.metadata.toHex(). A user found an edgecase where it fails to decode when using the metadata inside the unsigned tx in order to create a new registry.

I wrote a reproducible example in pure polkadot-js to get some feedback on the behavior.
My main question is, is it expected behavior that after using setMetadata on the registry to not be able to use that metadata again to create a new TypeRegistry.

Reproducible example

const { ApiPromise, WsProvider } = require('@polkadot/api');
const { TypeRegistry, Metadata } = require('@polkadot/types');

const main = async () => {
    const api = new ApiPromise({
        provider: new WsProvider('wss://kusama-rpc.polkadot.io')
    });

    await api.isReady

    const metadata = await api.rpc.state.getMetadata();
    const registry = new TypeRegistry()
    registry.setMetadata(metadata)

    const newMetadata = new Metadata(registry, registry.metadata.toHex()) // Fails here
    const newRegistry = new TypeRegistry()
    newRegistry.setMetadata(newMetadata)
}

main().then(() => process.exit()).catch(err => console.log(err)).finally(() => process.exit())

Error Message

Error: decodeU8a: failed at 0x310b000c1c73705f636f726518637279… on magicNumber: u32:: MagicNumber mismatch: expected 0x6174656d, found 0x0c000b31

Enviornment

Can confirm the above using both 6.10.3, 6.11.1 for @polkadot/api

[lcovar]

Running into this same error. Any suggestions for a fix?

[byhow]

I also got in this same issue but with a different magic number mismatch found 0x0000045d. I built the metadata and the transaction from the latest westend and the polkadot/substrate packages, node version 16.13.0.

[jacogr]

The metadata from registry is not the full metadata, it is a converted/as-lastest metadata.

This is the difference -

• For the original from-runtime metadata, it is basically a magic number, followed by an enum (0-14) and then the data
• For the asLatest metadata, it is converted to V14 (which is the latest) and has no magic number or version identifier

So the failure above is expected - it cannot find the magic in the metadata that the registry exposes since it has been transformed. If you want to keep track of the original on-chain metadata, you need to do so yourself. The Registry only uses the transformed metadata (aka it works on a consistent interface with a single known latest version)

See https://github.com/polkadot-js/api/blob/master/packages/types/src/types/registry.ts#L108 vs https://github.com/polkadot-js/api/blob/master/packages/types/src/types/registry.ts#L147

The set is Metadata the getter is MetadataLatest, they are not the same types.

Metadata construction would expect either a hex-representation of Metadata or an object of this type - this is where it fails, since the input is neither.

EDIT: Metadata could possibly also allow construction via MetadatLatest (PRs welcome, it basically can do an instanceof or equivalent check and then bypass quite a bit of logic, "some" tweaks required), but it will always have issues with a hex representation of MetadataLatest since it bytes can effectively be anything.

EDIT2: The alternative (looking at your usage) is for registry setMetadata to accept either Metadata or MetadataLatest - since it immediately converts the former when setting, see https://github.com/polkadot-js/api/blob/master/packages/types/src/create/registry.ts#L487-L488 (so instanceof or equivalent check and then do the internal #metadata set as required, once again PRs welcome if this solves your usecase)

TL;DR All expected, these are different types of the input to the registry via the output from the registry (Here is an old comment around the internal conversion - #4058 (comment))

[TarikGul]

@jacogr Thanks for the amazing explanation :)

From a use case perspective, I don't see my above example being a proper implementation either, but it did make me think about the expendability of the metadata within the registry which I ended up learning a ton about residually.

Metadata construction would expect either a hex-representation of Metadata or an object of this type.

This makes sense, while also looking back at the previous discussion we had via the comment you linked it's all coming full circle. In a case like this as well, there are so many options on how to handle the Metadata one is dealing with that there was really no reason for me to resort to using the TypeRegistry as a 'cache'/source-of-truth for the metadata. But instead to use it for what it's really good at doing, which is creating types, classes etc.

EDIT2: The Alternative (looking at your usage) is for registry setMetadata to accept either Metadata or MetadataLatest - since it immediately converts the former when setting, see https://github.com/polkadot-js/api/blob/master/packages/types/src/create/registry.ts#L487-L488 (so instanceof check and then do the internal #metadata set as required, once again Prs welcome if this solves your usecase)

That being said, I have been meaning and wanting to add meaningful contributions to polkadot-js for a while now, so whether it's something related to this (if I can find a good justifiable reason for our use case), or something else ill start jumping on it in my free time!

Seriously thanks for taking your time for the great explanation, I left this with a much better understanding.

Closing the issue as it's been resolved.

[polkadot-js-bot]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue if you think you have a related problem or query.