Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

StorageWeightReclaim: Fix issue when underestimating refund. #5273

Merged
merged 4 commits into from
Aug 9, 2024

Conversation

gui1117
Copy link
Contributor

@gui1117 gui1117 commented Aug 7, 2024

The code do reduce or increase the weight by comparing benchmarked_weight and consumed_weight.

But benchmarked_weight is the pre dispatch weight. not the post dispatch weight that is actually written into the block weight by CheckWeight.

So in case the consumed weight was: pre dispatch weight > consumed weight > post dispatch weight then the reclaim code was reducing the block weight instead of increasing it.

Might explain this issue even better #5229

@skunert
@s0me0ne-unkn0wn

@skunert
Copy link
Contributor

skunert commented Aug 7, 2024

Ouch 🤦.

Good find! @s0me0ne-unkn0wn can you throw your verification zombienet on this branch and see if it changes things?

@skunert
Copy link
Contributor

skunert commented Aug 7, 2024

I am thinking about the impact this bug has.

The actual_weight in the post info will be useful if you know during execution that you take a specific codepath and basically return only a subset of the weight. However that part that you return as actual_weight should still be benchmarked and an overestimation right? Not saying it can't happen, but my impression is that in most cases the actual_weight should still be bigger than the real used proof size on the node.

@s0me0ne-unkn0wn
Copy link
Contributor

From my pov, it doesn't improve the situation, probably even making it a bit worse.

Runtime block weight info:

{
  normal: {
    refTime: 1,230,370,849,000
    proofSize: 2,923,186
  }
  operational: {
    refTime: 0
    proofSize: 0
  }
  mandatory: {
    refTime: 5,203,319,000
    proofSize: 77,900
  }
}

Registered by node: PoV size { header: 0.216796875kb, extrinsics: 626.0771484375kb, storage_proof: 5043.0947265625kb }

Something's wrong, although I haven't run with full logs yet, maybe it's something obvious. Will try later today.

@skunert
Copy link
Contributor

skunert commented Aug 9, 2024

probably even making it a bit worse

How is it worse? Looks unchanged to the results you posted in the past.

The scenario here is only occuring if the PostDispatchInfo contains an actual_weight that is lower than the consumed proof size. As far as I see the current situation where we miss out measuring the proof size of some of the SEs makes this even more unlikely to surface.

@skunert skunert added the T1-FRAME This PR/Issue is related to core FRAME, the framework. label Aug 9, 2024
@s0me0ne-unkn0wn
Copy link
Contributor

How is it worse? Looks unchanged to the results you posted in the past.

Yes, you're right. I forgot that in the previous round of tests, the collator's PoV limit was raised to 6 Mb, so these numbers seemed a bit too low to me.

And IIUC, this PR only changes how unspent is handled, and I've never observed non-zero unspent in my tests 🤷

@skunert skunert added this pull request to the merge queue Aug 9, 2024
Merged via the queue into paritytech:master with commit 862860e Aug 9, 2024
171 of 176 checks passed
@gui1117 gui1117 deleted the gui-fix-storage-reclaim branch August 10, 2024 00:20
fellowship-merge-bot bot pushed a commit to polkadot-fellows/runtimes that referenced this pull request Aug 13, 2024
I propose to move PoV-reclaim to the next release.

It can happen that the amount to reclaim is overestimated
paritytech/polkadot-sdk#5229.

Improvements (paritytech/polkadot-sdk#5281,
paritytech/polkadot-sdk#5273) and a proper fix
(paritytech/polkadot-sdk#5234) are in the works,
but should be tested again.

I expect the chances that the system-chains run into this scenario to be
low, but it does not look like activity is so high that the inclusion of
reclaim is urgent.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
skunert pushed a commit that referenced this pull request Aug 13, 2024
The code do reduce or increase the weight by comparing
`benchmarked_weight` and `consumed_weight`.

But `benchmarked_weight` is the pre dispatch weight. not the post
dispatch weight that is actually written into the block weight by
`CheckWeight`.

So in case the consumed weight was: `pre dispatch weight > consumed
weight > post dispatch weight` then the reclaim code was reducing the
block weight instead of increasing it.

Might explain this issue even better
#5229

@skunert
@s0me0ne-unkn0wn

(cherry picked from commit 862860e)
EgorPopelyaev pushed a commit that referenced this pull request Aug 14, 2024
Backports #5273 & #5281

---------

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
skunert pushed a commit that referenced this pull request Aug 15, 2024
The code do reduce or increase the weight by comparing
`benchmarked_weight` and `consumed_weight`.

But `benchmarked_weight` is the pre dispatch weight. not the post
dispatch weight that is actually written into the block weight by
`CheckWeight`.

So in case the consumed weight was: `pre dispatch weight > consumed
weight > post dispatch weight` then the reclaim code was reducing the
block weight instead of increasing it.

Might explain this issue even better
#5229

@skunert
@s0me0ne-unkn0wn

(cherry picked from commit 862860e)
EgorPopelyaev pushed a commit that referenced this pull request Aug 15, 2024
Backports #5273 &
#5281

---------

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
dharjeezy pushed a commit to dharjeezy/polkadot-sdk that referenced this pull request Aug 28, 2024
…ech#5273)

The code do reduce or increase the weight by comparing
`benchmarked_weight` and `consumed_weight`.

But `benchmarked_weight` is the pre dispatch weight. not the post
dispatch weight that is actually written into the block weight by
`CheckWeight`.

So in case the consumed weight was: `pre dispatch weight > consumed
weight > post dispatch weight` then the reclaim code was reducing the
block weight instead of increasing it.

Might explain this issue even better
paritytech#5229

@skunert 
@s0me0ne-unkn0wn
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T1-FRAME This PR/Issue is related to core FRAME, the framework.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants