Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Storehouse] Update committer #5029

Merged
merged 15 commits into from
Nov 22, 2023
Merged

[Storehouse] Update committer #5029

merged 15 commits into from
Nov 22, 2023

Conversation

zhangchiqing
Copy link
Member

@zhangchiqing zhangchiqing commented Nov 16, 2023

Working towards #4998

This PR updates committer to use snapshot instead of commitment when commit delta

@zhangchiqing zhangchiqing changed the base branch from master to leo/storehouse-extending-block-snapshot November 16, 2023 23:00
@codecov-commenter
Copy link

codecov-commenter commented Nov 16, 2023

Codecov Report

Attention: 12 lines in your changes are missing coverage. Please review.

Comparison is base (538e72e) 56.20% compared to head (086b714) 56.20%.
Report is 8 commits behind head on master.

Files Patch % Lines
engine/execution/computation/committer/noop.go 0.00% 5 Missing ⚠️
...e/execution/storehouse/executing_block_snapshot.go 40.00% 2 Missing and 1 partial ⚠️
engine/execution/state/state.go 83.33% 2 Missing ⚠️
fvm/storage/snapshot/execution_snapshot.go 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5029      +/-   ##
==========================================
- Coverage   56.20%   56.20%   -0.01%     
==========================================
  Files         975      975              
  Lines       90714    90754      +40     
==========================================
+ Hits        50990    51006      +16     
- Misses      35913    35937      +24     
  Partials     3811     3811              
Flag Coverage Δ
unittests 56.20% <75.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@zhangchiqing zhangchiqing force-pushed the leo/storehouse-extending-block-snapshot branch from 27715b2 to 904bed0 Compare November 18, 2023 00:42
Base automatically changed from leo/storehouse-extending-block-snapshot to master November 18, 2023 02:24
require.Equal(t, []uint8(expectedProof), proof)
require.True(t, expectedTrieUpdate.Equals(trieUpdate))

// TOOD(leo): verify ledger.Set and ledger.Prove are called and received expected arguments
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How to do this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can put a function inside the mock like so:

l.On("Set", mock.Anything).
			Return(func(/*args with types*/) /*return vals*/ {
                              // assert whatever you want
			     return ledger.State(endState), expectedTrieUpdate, nil
			}).
			Once()

@zhangchiqing zhangchiqing marked this pull request as ready for review November 18, 2023 02:42
@zhangchiqing zhangchiqing requested review from koko1123, sideninja and peterargue and removed request for ramtinms November 18, 2023 02:43
@@ -58,6 +58,10 @@ func (s *ExecutingBlockSnapshot) getFromUpdates(id flow.RegisterID) (flow.Regist
// Usually it's used to create a new storage snapshot at the next executed collection.
// The registerUpdates contains the register updates at the executed collection.
func (s *ExecutingBlockSnapshot) Extend(newCommit flow.StateCommitment, updates map[flow.RegisterID]flow.RegisterValue) execution.ExtendableStorageSnapshot {
if len(updates) == 0 {
return s
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

whats the reasoning here? I'm not sure I understand.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will add comments, the idea is that if there is. no updates, it is not necessary to wrap with a snapshot that has no update, instead, just return the original snapshot itself, since it is able to return the same register.

Comment on lines 226 to +227
UpdatedRegisters() flow.RegisterEntries
UpdatedRegisterSet() map[flow.RegisterID]flow.RegisterValue
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function below uses both of these. I think that is unnecessary. We could only have UpdatedRegisterSet() map[flow.RegisterID]flow.RegisterValue (which means we could just call it UpdatedRegisters() map[flow.RegisterID]flow.RegisterValue)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, however it requires the caller to convert into a slice. And we probably don't want the caller to decide the order, the order of updates should be deterministic, that's UpdatedRegisters sort the slice internally before returning.

What about returning both map and slice, since both of them are used below?
It's possible, but I'm just afraid it's a bit confusing, returning the same data in two forms in one API. That's why I created a separate API for returning different form of the same data.

Thoughts?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RegisterUpdatesHolder is only used here and the only use of UpdatedRegisters is then converted into a map anyway, so order is not conserved.

I'm just not sure the function below (CommitDelta) needs to use both these interface methods.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the order is actually conserved by keys, values := RegisterEntriesToKeysValues(updatedRegisters)

The keys and values are ordered, which then be used to create trieUpdate by

	newState, trieUpdate, err := ldg.Set(update)

The trieUpdate contains the ordered list of paths and payloads, the order has to be deterministic, otherwise it will produce different execution data id in the execution result.

@zhangchiqing zhangchiqing added this pull request to the merge queue Nov 22, 2023
Merged via the queue into master with commit 0ba81ff Nov 22, 2023
54 checks passed
@zhangchiqing zhangchiqing deleted the leo/storehouse-committer branch November 22, 2023 17:09
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants