Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle event encoding error gracefully (Port of 2562) #2565

Merged
merged 28 commits into from
Oct 20, 2022
Merged

Conversation

m4ksio
Copy link
Contributor

@m4ksio m4ksio commented Jun 6, 2022

Originally this was a simple port, but it was unmerged for too long and I had to start from scratch.
I

@github-actions
Copy link
Contributor

github-actions bot commented Jun 6, 2022

FVM Benchstat comparison

This branch with compared with the base branch onflow:master commit 56d7658

The command (for i in {1..10}; do go test ./fvm ./engine/execution/computation --bench . --tags relic -shuffle=on --benchmem --run ^$; done) was used.

Collapsed results for better readability

old.txtnew.txt
time/opdelta
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeNFTBatchTransfer-2128ms ± 6%126ms ± 3%~(p=0.113 n=9+10)
RuntimeTransaction/reference_tx-234.9ms ± 5%34.7ms ± 5%~(p=0.730 n=9+9)
RuntimeTransaction/convert_int_to_string-239.3ms ±14%38.3ms ±16%~(p=0.353 n=10+10)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-238.2ms ± 7%36.8ms ± 2%~(p=0.050 n=9+9)
RuntimeTransaction/get_signer_address-235.8ms ± 8%37.2ms ±17%~(p=0.400 n=9+10)
RuntimeTransaction/get_public_account-238.7ms ± 6%38.3ms ± 5%~(p=0.730 n=9+9)
RuntimeTransaction/get_account_and_get_balance-2336ms ± 2%333ms ± 4%~(p=0.497 n=9+10)
RuntimeTransaction/get_account_and_get_available_balance-2291ms ± 5%290ms ± 2%~(p=0.604 n=10+9)
RuntimeTransaction/get_account_and_get_storage_used-240.9ms ±10%40.6ms ± 9%~(p=0.912 n=10+10)
RuntimeTransaction/get_account_and_get_storage_capacity-2260ms ± 3%256ms ± 5%~(p=0.340 n=9+9)
RuntimeTransaction/get_signer_vault-243.0ms ± 5%41.5ms ± 4%~(p=0.075 n=10+10)
RuntimeTransaction/get_signer_receiver-252.7ms ± 5%51.7ms ± 9%~(p=0.356 n=9+10)
RuntimeTransaction/transfer_tokens-2212ms ± 5%209ms ± 6%~(p=0.529 n=10+10)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-243.0ms ±12%42.5ms ± 5%~(p=0.631 n=10+10)
RuntimeTransaction/load_and_save_long_string_on_signers_address-288.6ms ± 8%87.8ms ± 5%~(p=0.842 n=9+10)
RuntimeTransaction/create_new_account-2916ms ± 2%908ms ± 9%~(p=0.356 n=9+10)
RuntimeTransaction/call_empty_contract_function-238.5ms ±12%39.0ms ± 7%~(p=0.356 n=9+10)
RuntimeTransaction/emit_event-251.0ms ± 8%51.5ms ± 7%~(p=0.684 n=10+10)
RuntimeTransaction/borrow_array_from_storage-2142ms ± 6%142ms ± 8%~(p=0.968 n=9+10)
RuntimeTransaction/copy_array_from_storage-2148ms ± 5%147ms ± 2%~(p=0.122 n=10+8)
pkg:github.com/onflow/flow-go/engine/execution/computation goos:linux goarch:amd64
ComputeBlock/16/cols/128/txes-25.20s ± 3%5.13s ± 2%~(p=0.136 n=9+9)
 
alloc/opdelta
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeTransaction/emit_event-239.5MB ± 4%40.4MB ± 6%+2.14%(p=0.035 n=10+10)
RuntimeNFTBatchTransfer-255.5MB ± 3%54.4MB ± 6%~(p=0.094 n=9+9)
RuntimeTransaction/reference_tx-234.9MB ± 2%34.9MB ± 2%~(p=0.853 n=10+10)
RuntimeTransaction/convert_int_to_string-235.6MB ± 3%35.1MB ± 3%~(p=0.165 n=10+10)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-236.0MB ± 5%35.7MB ± 4%~(p=0.393 n=10+10)
RuntimeTransaction/get_signer_address-235.7MB ± 2%35.3MB ± 4%~(p=0.105 n=10+10)
RuntimeTransaction/get_public_account-236.7MB ± 7%36.6MB ± 3%~(p=0.356 n=10+9)
RuntimeTransaction/get_account_and_get_balance-2130MB ± 2%130MB ± 2%~(p=0.796 n=10+10)
RuntimeTransaction/get_account_and_get_available_balance-2109MB ± 3%110MB ± 0%~(p=0.962 n=10+7)
RuntimeTransaction/get_account_and_get_storage_used-237.4MB ± 5%37.0MB ± 4%~(p=0.315 n=10+10)
RuntimeTransaction/get_account_and_get_storage_capacity-2106MB ± 4%105MB ± 4%~(p=0.436 n=10+10)
RuntimeTransaction/get_signer_vault-237.0MB ± 3%36.6MB ± 3%~(p=0.218 n=10+10)
RuntimeTransaction/get_signer_receiver-240.4MB ± 3%40.1MB ± 7%~(p=0.631 n=10+10)
RuntimeTransaction/transfer_tokens-283.3MB ± 3%83.9MB ± 4%~(p=0.796 n=10+10)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-236.4MB ± 2%36.8MB ± 4%~(p=0.280 n=10+10)
RuntimeTransaction/load_and_save_long_string_on_signers_address-255.4MB ± 3%55.8MB ± 3%~(p=0.481 n=10+10)
RuntimeTransaction/create_new_account-2198MB ± 2%197MB ± 3%~(p=0.218 n=10+10)
RuntimeTransaction/call_empty_contract_function-235.8MB ± 3%35.9MB ± 3%~(p=0.481 n=10+10)
RuntimeTransaction/borrow_array_from_storage-270.1MB ± 2%69.5MB ± 2%~(p=0.123 n=10+10)
pkg:github.com/onflow/flow-go/engine/execution/computation goos:linux goarch:amd64
ComputeBlock/16/cols/128/txes-21.28GB ± 1%1.28GB ± 1%~(p=0.353 n=10+10)
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeTransaction/copy_array_from_storage-282.8MB ± 2%81.2MB ± 3%−1.92%(p=0.007 n=10+10)
 
allocs/opdelta
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeNFTBatchTransfer-2301k ± 1%301k ± 1%~(p=0.516 n=10+10)
RuntimeTransaction/get_account_and_get_balance-21.52M ± 0%1.52M ± 0%~(p=0.128 n=9+10)
RuntimeTransaction/get_account_and_get_storage_capacity-21.23M ± 0%1.23M ± 0%−0.00%(p=0.025 n=10+10)
RuntimeTransaction/get_account_and_get_available_balance-21.36M ± 0%1.36M ± 0%−0.00%(p=0.001 n=10+10)
RuntimeTransaction/borrow_array_from_storage-2347k ± 0%347k ± 0%−0.01%(p=0.003 n=8+10)
RuntimeTransaction/copy_array_from_storage-2303k ± 0%303k ± 0%−0.01%(p=0.000 n=10+10)
RuntimeTransaction/load_and_save_long_string_on_signers_address-2235k ± 0%235k ± 0%−0.01%(p=0.004 n=10+10)
RuntimeTransaction/get_signer_receiver-2205k ± 0%204k ± 0%−0.02%(p=0.000 n=10+10)
RuntimeTransaction/call_empty_contract_function-2105k ± 0%105k ± 0%−0.02%(p=0.000 n=10+10)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-2133k ± 0%133k ± 0%−0.02%(p=0.000 n=8+10)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-2113k ± 0%113k ± 0%−0.03%(p=0.000 n=10+10)
RuntimeTransaction/get_account_and_get_storage_used-2135k ± 0%135k ± 0%−0.03%(p=0.000 n=10+10)
RuntimeTransaction/get_signer_vault-2133k ± 0%133k ± 0%−0.03%(p=0.000 n=10+10)
RuntimeTransaction/get_public_account-2115k ± 0%115k ± 0%−0.03%(p=0.000 n=10+10)
RuntimeTransaction/get_signer_address-294.1k ± 0%94.1k ± 0%−0.03%(p=0.000 n=10+9)
RuntimeTransaction/convert_int_to_string-2102k ± 0%102k ± 0%−0.03%(p=0.000 n=10+10)
RuntimeTransaction/reference_tx-289.9k ± 0%89.9k ± 0%−0.03%(p=0.000 n=10+10)
pkg:github.com/onflow/flow-go/engine/execution/computation goos:linux goarch:amd64
ComputeBlock/16/cols/128/txes-219.9M ± 0%19.9M ± 0%−0.05%(p=0.000 n=10+10)
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeTransaction/create_new_account-22.57M ± 0%2.57M ± 0%−0.12%(p=0.000 n=9+10)
RuntimeTransaction/transfer_tokens-2862k ± 0%860k ± 0%−0.23%(p=0.000 n=10+10)
RuntimeTransaction/emit_event-2143k ± 0%142k ± 0%−0.71%(p=0.000 n=10+9)
 
computationdelta
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeTransaction/reference_tx-2202 ± 0%202 ± 0%~(all equal)
RuntimeTransaction/convert_int_to_string-2402 ± 0%402 ± 0%~(all equal)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-2502 ± 0%502 ± 0%~(all equal)
RuntimeTransaction/get_signer_address-2302 ± 0%302 ± 0%~(all equal)
RuntimeTransaction/get_public_account-2402 ± 0%402 ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_balance-21.00k ± 0%1.00k ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_available_balance-22.60k ± 0%2.60k ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_storage_used-2402 ± 0%402 ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_storage_capacity-21.30k ± 0%1.30k ± 0%~(all equal)
RuntimeTransaction/get_signer_vault-2402 ± 0%402 ± 0%~(all equal)
RuntimeTransaction/get_signer_receiver-2602 ± 0%602 ± 0%~(all equal)
RuntimeTransaction/transfer_tokens-23.50k ± 0%3.50k ± 0%~(all equal)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-2602 ± 0%602 ± 0%~(all equal)
RuntimeTransaction/load_and_save_long_string_on_signers_address-2602 ± 0%602 ± 0%~(all equal)
RuntimeTransaction/create_new_account-2202 ± 0%202 ± 0%~(all equal)
RuntimeTransaction/call_empty_contract_function-2402 ± 0%402 ± 0%~(all equal)
RuntimeTransaction/emit_event-2602 ± 0%602 ± 0%~(all equal)
RuntimeTransaction/borrow_array_from_storage-22.60k ± 0%2.60k ± 0%~(all equal)
RuntimeTransaction/copy_array_from_storage-22.60k ± 0%2.60k ± 0%~(all equal)
 
interactionsdelta
pkg:github.com/onflow/flow-go/fvm goos:linux goarch:amd64
RuntimeTransaction/reference_tx-244.3k ± 0%44.3k ± 0%~(all equal)
RuntimeTransaction/convert_int_to_string-244.3k ± 0%44.3k ± 0%~(all equal)
RuntimeTransaction/convert_int_to_string_and_concatenate_it-244.3k ± 0%44.3k ± 0%~(all equal)
RuntimeTransaction/get_signer_address-244.3k ± 0%44.3k ± 0%~(all equal)
RuntimeTransaction/get_public_account-244.3k ± 0%44.3k ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_balance-216.7M ± 0%16.7M ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_available_balance-25.13M ± 0%5.13M ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_storage_used-244.3k ± 0%44.3k ± 0%~(all equal)
RuntimeTransaction/get_account_and_get_storage_capacity-25.13M ± 0%5.13M ± 0%~(all equal)
RuntimeTransaction/get_signer_vault-244.5k ± 0%44.5k ± 0%~(all equal)
RuntimeTransaction/get_signer_receiver-244.9k ± 0%44.9k ± 0%~(all equal)
RuntimeTransaction/transfer_tokens-245.2k ± 0%45.2k ± 0%~(all equal)
RuntimeTransaction/load_and_save_empty_string_on_signers_address-244.4k ± 0%44.4k ± 0%~(all equal)
RuntimeTransaction/load_and_save_long_string_on_signers_address-247.2k ± 0%47.2k ± 0%~(all equal)
RuntimeTransaction/create_new_account-28.39M ± 0%8.39M ± 0%~(all equal)
RuntimeTransaction/call_empty_contract_function-244.5k ± 0%44.5k ± 0%~(all equal)
RuntimeTransaction/emit_event-244.5k ± 0%44.5k ± 0%~(all equal)
RuntimeTransaction/borrow_array_from_storage-249.7k ± 0%49.7k ± 0%~(all equal)
RuntimeTransaction/copy_array_from_storage-249.7k ± 0%49.7k ± 0%~(all equal)
 
us/txdelta
pkg:github.com/onflow/flow-go/engine/execution/computation goos:linux goarch:amd64
ComputeBlock/16/cols/128/txes-22.54k ± 3%2.50k ± 2%~(p=0.136 n=9+9)
 

@codecov-commenter
Copy link

codecov-commenter commented Jun 6, 2022

Codecov Report

Merging #2565 (633d0a4) into master (56d7658) will decrease coverage by 3.30%.
The diff coverage is 56.66%.

@@            Coverage Diff             @@
##           master    #2565      +/-   ##
==========================================
- Coverage   55.59%   52.29%   -3.31%     
==========================================
  Files         757      555     -202     
  Lines       69971    51648   -18323     
==========================================
- Hits        38899    27008   -11891     
+ Misses      27881    22422    -5459     
+ Partials     3191     2218     -973     
Flag Coverage Δ
unittests 52.29% <56.66%> (-3.31%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
fvm/context.go 57.50% <0.00%> (-1.86%) ⬇️
fvm/errors/codes.go 100.00% <ø> (ø)
fvm/errors/errors.go 57.14% <0.00%> (-2.44%) ⬇️
fvm/environment/event_emitter.go 70.14% <66.66%> (+1.26%) ⬆️
fvm/environment/event_encoder.go 83.33% <83.33%> (ø)
insecure/orchestrator/mock_corrupt_network.go 80.30% <0.00%> (-3.04%) ⬇️
fvm/environment/contract_updater.go 66.88% <0.00%> (ø)
engine/collection/synchronization/engine.go 68.97% <0.00%> (ø)
module/dkg/broker.go
network/queue/eventPriority.go
... and 203 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Copy link
Member

@ramtinms ramtinms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for porting this change back

@Kay-Zee
Copy link
Member

Kay-Zee commented Jul 14, 2022

bors merge

bors bot added a commit that referenced this pull request Jul 14, 2022
2565: Port of 2562 r=Kay-Zee a=m4ksio



Co-authored-by: Maks Pawlak <120831+m4ksio@users.noreply.github.com>
Co-authored-by: Ramtin M. Seraj <ramtinms@users.noreply.github.com>
@bors
Copy link
Contributor

bors bot commented Jul 14, 2022

Build failed:

@m4ksio
Copy link
Contributor Author

m4ksio commented Jul 26, 2022

bors merge

bors bot added a commit that referenced this pull request Jul 26, 2022
2565: Port of 2562 r=m4ksio a=m4ksio



Co-authored-by: Maks Pawlak <120831+m4ksio@users.noreply.github.com>
Co-authored-by: Ramtin M. Seraj <ramtinms@users.noreply.github.com>
Co-authored-by: Maks Pawlak <makspawlak@Makss-MacBook-Pro.local>
@bors
Copy link
Contributor

bors bot commented Jul 26, 2022

Build failed:

@m4ksio m4ksio changed the title Port of 2562 Handle event encoding error gracefully (Port of 2562) Oct 19, 2022
}
}

func (e *CadenceEventEncoder) Encode(event cadence.Event) ([]byte, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just call return jsoncdc.Encode(event) here? Since you end up resetting the buffer / copying the payload on each Encode call, there no benefit in pre-allocating a buffer / encoder.

Copy link
Contributor Author

@m4ksio m4ksio Oct 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reusing the buffer should give some speedeup, as resetting keeps underlaying bytes, but this seemed to be rather insignificant. So I did a benchmark and looks like there is quite a difference.

BenchmarkEncodingBuffer-16               1306316               949.6 ns/op
BenchmarkEncodingBuffer-16               1230076               944.8 ns/op
BenchmarkEncodingBuffer-16               1282364               941.2 ns/op
BenchmarkEncodingBuffer-16               1311118               900.7 ns/op
BenchmarkEncodingBufferV2-16             1468462               807.6 ns/op
BenchmarkEncodingBufferV2-16             1501274               821.0 ns/op
BenchmarkEncodingBufferV2-16             1456159               812.9 ns/op
BenchmarkEncodingBufferV2-16             1464187               818.2 ns/op
BenchmarkEncodingBufferV2-16             1410309               829.4 ns/op
PASS
ok      github.com/onflow/flow-go/fvm/environment       20.927

V2 is the new version. I think difference would be even more visible for larger events.

Copy link
Member

@ramtinms ramtinms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@m4ksio m4ksio merged commit d4be6e5 into master Oct 20, 2022
@m4ksio m4ksio deleted the m4ksio/port-2562 branch October 20, 2022 17:42
@m4ksio
Copy link
Contributor Author

m4ksio commented Oct 24, 2022

Closes #2563

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants