feat: async shuffling refactor #6938

matthewkeil · 2024-07-08T21:22:49Z

Motivation

Replaces #6521

Move calculation of next shuffling to async to get it off of critical path during epoch transition. There is a full second during epoch transition used to calculate the epochCtx.nextShuffling and that can be moved to an async process. Refactored a few pieces of the EpochCache to make this work and will continue under a separate PR for creating a worker that moves this calculation to a worker thread. Also will investigate using the rust shuffling implementation. Can maybe tune further using a worker thread that NICE to interleave the long calculation into thread idle time which could be ideal.

Description

Create IShufflingCache to pass shuffling cache to EpochCache so its available within CachedBeaconState
Keep ShufflingCache in beacon-node for further updates mentioned above
Remove calculation of nextShuffling from critical path during epoch transitions
Remove shufflingCache.processState from ShufflingCache and update process flow to no longer require it
Remove ShufflingGetter and use ShufflingCache directly

TODO's Before Review
[ ] - Investigate removal of MockShufflingCache fully (may require moving CachedBeaconState tests to beacon-node
[ ] - Verify all sim/e2e and other long running tests
[ ] - Confirm spec compliance

codecov · 2024-07-09T20:05:52Z

Codecov Report

Attention: Patch coverage is 35.88850% with 184 lines in your changes missing coverage. Please review.

Project coverage is 50.84%. Comparing base (19ac678) to head (88d3105).
Report is 24 commits behind head on unstable.

Additional details and impacted files

@@             Coverage Diff              @@
##           unstable    #6938      +/-   ##
============================================
+ Coverage     49.35%   50.84%   +1.49%     
============================================
  Files           592      597       +5     
  Lines         39293    39835     +542     
  Branches       2248     2256       +8     
============================================
+ Hits          19392    20254     +862     
+ Misses        19860    19581     -279     
+ Partials         41        0      -41

packages/state-transition/src/cache/epochCache.ts

github-actions · 2024-07-11T23:52:54Z

Performance Report

✔️ no performance regression detected

Full benchmark results

Benchmark suite	Current: `676a560`	Previous: `bb40ef7`	Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc	1.9932 ms/op	1.9819 ms/op	1.01
getPubkeys - validatorsArr - req 1000 vs - 250000 vc	54.366 us/op	48.475 us/op	1.12
BLS verify - blst	962.18 us/op	1.0266 ms/op	0.94
BLS verifyMultipleSignatures 3 - blst	1.2601 ms/op	1.3525 ms/op	0.93
BLS verifyMultipleSignatures 8 - blst	1.9258 ms/op	2.1102 ms/op	0.91
BLS verifyMultipleSignatures 32 - blst	5.6387 ms/op	6.6228 ms/op	0.85
BLS verifyMultipleSignatures 64 - blst	10.825 ms/op	10.977 ms/op	0.99
BLS verifyMultipleSignatures 128 - blst	17.853 ms/op	17.663 ms/op	1.01
BLS deserializing 10000 signatures	692.82 ms/op	682.25 ms/op	1.02
BLS deserializing 100000 signatures	6.9241 s/op	6.9150 s/op	1.00
BLS verifyMultipleSignatures - same message - 3 - blst	951.01 us/op	1.0821 ms/op	0.88
BLS verifyMultipleSignatures - same message - 8 - blst	1.0836 ms/op	1.1089 ms/op	0.98
BLS verifyMultipleSignatures - same message - 32 - blst	1.8144 ms/op	1.8012 ms/op	1.01
BLS verifyMultipleSignatures - same message - 64 - blst	2.7124 ms/op	2.7281 ms/op	0.99
BLS verifyMultipleSignatures - same message - 128 - blst	4.3748 ms/op	4.3597 ms/op	1.00
BLS aggregatePubkeys 32 - blst	19.917 us/op	19.575 us/op	1.02
BLS aggregatePubkeys 128 - blst	71.037 us/op	70.147 us/op	1.01
notSeenSlots=1 numMissedVotes=1 numBadVotes=10	69.567 ms/op	70.245 ms/op	0.99
notSeenSlots=1 numMissedVotes=0 numBadVotes=4	61.976 ms/op	69.885 ms/op	0.89
notSeenSlots=2 numMissedVotes=1 numBadVotes=10	37.267 ms/op	39.012 ms/op	0.96
getSlashingsAndExits - default max	111.68 us/op	98.465 us/op	1.13
getSlashingsAndExits - 2k	293.35 us/op	304.24 us/op	0.96
proposeBlockBody type=full, size=empty	5.9039 ms/op	6.1662 ms/op	0.96
isKnown best case - 1 super set check	320.00 ns/op	331.00 ns/op	0.97
isKnown normal case - 2 super set checks	301.00 ns/op	304.00 ns/op	0.99
isKnown worse case - 16 super set checks	288.00 ns/op	312.00 ns/op	0.92
InMemoryCheckpointStateCache - add get delete	2.9900 us/op	2.9090 us/op	1.03
updateUnfinalizedPubkeys - updating 10 pubkeys	1.2484 ms/op	1.5418 ms/op	0.81
updateUnfinalizedPubkeys - updating 100 pubkeys	4.1333 ms/op	4.2937 ms/op	0.96
updateUnfinalizedPubkeys - updating 1000 pubkeys	55.503 ms/op	56.575 ms/op	0.98
validate api signedAggregateAndProof - struct	1.8255 ms/op	1.6026 ms/op	1.14
validate gossip signedAggregateAndProof - struct	1.6136 ms/op	1.6909 ms/op	0.95
validate gossip attestation - vc 640000	1.0929 ms/op	1.0663 ms/op	1.02
batch validate gossip attestation - vc 640000 - chunk 32	149.01 us/op	149.63 us/op	1.00
batch validate gossip attestation - vc 640000 - chunk 64	133.36 us/op	131.01 us/op	1.02
batch validate gossip attestation - vc 640000 - chunk 128	123.74 us/op	125.54 us/op	0.99
batch validate gossip attestation - vc 640000 - chunk 256	119.13 us/op	124.48 us/op	0.96
pickEth1Vote - no votes	1.2161 ms/op	1.2091 ms/op	1.01
pickEth1Vote - max votes	7.9620 ms/op	8.8825 ms/op	0.90
pickEth1Vote - Eth1Data hashTreeRoot value x2048	14.555 ms/op	14.882 ms/op	0.98
pickEth1Vote - Eth1Data hashTreeRoot tree x2048	20.240 ms/op	21.231 ms/op	0.95
pickEth1Vote - Eth1Data fastSerialize value x2048	527.16 us/op	579.61 us/op	0.91
pickEth1Vote - Eth1Data fastSerialize tree x2048	3.0579 ms/op	4.5053 ms/op	0.68
bytes32 toHexString	541.00 ns/op	506.00 ns/op	1.07
bytes32 Buffer.toString(hex)	271.00 ns/op	272.00 ns/op	1.00
bytes32 Buffer.toString(hex) from Uint8Array	461.00 ns/op	465.00 ns/op	0.99
bytes32 Buffer.toString(hex) + 0x	277.00 ns/op	267.00 ns/op	1.04
Object access 1 prop	0.16900 ns/op	0.15600 ns/op	1.08
Map access 1 prop	0.13600 ns/op	0.13300 ns/op	1.02
Object get x1000	6.1960 ns/op	6.2810 ns/op	0.99
Map get x1000	6.7370 ns/op	6.9350 ns/op	0.97
Object set x1000	37.046 ns/op	39.276 ns/op	0.94
Map set x1000	26.217 ns/op	26.151 ns/op	1.00
Return object 10000 times	0.31110 ns/op	0.33060 ns/op	0.94
Throw Error 10000 times	3.4683 us/op	3.8156 us/op	0.91
toHex	175.05 ns/op	162.48 ns/op	1.08
Buffer.from	158.98 ns/op	148.25 ns/op	1.07
shared Buffer	95.212 ns/op	106.21 ns/op	0.90
fastMsgIdFn sha256 / 200 bytes	2.4470 us/op	2.4600 us/op	0.99
fastMsgIdFn h32 xxhash / 200 bytes	289.00 ns/op	288.00 ns/op	1.00
fastMsgIdFn h64 xxhash / 200 bytes	273.00 ns/op	295.00 ns/op	0.93
fastMsgIdFn sha256 / 1000 bytes	7.5250 us/op	7.5690 us/op	0.99
fastMsgIdFn h32 xxhash / 1000 bytes	420.00 ns/op	419.00 ns/op	1.00
fastMsgIdFn h64 xxhash / 1000 bytes	352.00 ns/op	377.00 ns/op	0.93
fastMsgIdFn sha256 / 10000 bytes	66.383 us/op	68.718 us/op	0.97
fastMsgIdFn h32 xxhash / 10000 bytes	1.9400 us/op	1.9820 us/op	0.98
fastMsgIdFn h64 xxhash / 10000 bytes	1.2390 us/op	1.3070 us/op	0.95
send data - 1000 256B messages	13.843 ms/op	14.693 ms/op	0.94
send data - 1000 512B messages	18.556 ms/op	20.004 ms/op	0.93
send data - 1000 1024B messages	28.172 ms/op	31.382 ms/op	0.90
send data - 1000 1200B messages	30.329 ms/op	33.126 ms/op	0.92
send data - 1000 2048B messages	34.640 ms/op	39.297 ms/op	0.88
send data - 1000 4096B messages	36.032 ms/op	37.175 ms/op	0.97
send data - 1000 16384B messages	81.065 ms/op	77.705 ms/op	1.04
send data - 1000 65536B messages	246.84 ms/op	245.52 ms/op	1.01
enrSubnets - fastDeserialize 64 bits	1.3510 us/op	1.2950 us/op	1.04
enrSubnets - ssz BitVector 64 bits	428.00 ns/op	468.00 ns/op	0.91
enrSubnets - fastDeserialize 4 bits	177.00 ns/op	211.00 ns/op	0.84
enrSubnets - ssz BitVector 4 bits	452.00 ns/op	473.00 ns/op	0.96
prioritizePeers score -10:0 att 32-0.1 sync 2-0	186.30 us/op	186.81 us/op	1.00
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25	195.71 us/op	188.93 us/op	1.04
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5	331.52 us/op	457.17 us/op	0.73
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75	532.52 us/op	546.38 us/op	0.97
prioritizePeers score 0:0 att 64-1 sync 4-1	941.13 us/op	1.0111 ms/op	0.93
array of 16000 items push then shift	1.7320 us/op	1.9126 us/op	0.91
LinkedList of 16000 items push then shift	8.0360 ns/op	10.118 ns/op	0.79
array of 16000 items push then pop	138.88 ns/op	167.66 ns/op	0.83
LinkedList of 16000 items push then pop	7.4320 ns/op	9.7150 ns/op	0.77
array of 24000 items push then shift	2.4781 us/op	2.9853 us/op	0.83
LinkedList of 24000 items push then shift	7.6500 ns/op	9.5450 ns/op	0.80
array of 24000 items push then pop	178.02 ns/op	196.25 ns/op	0.91
LinkedList of 24000 items push then pop	7.9020 ns/op	8.7520 ns/op	0.90
intersect bitArray bitLen 8	6.7960 ns/op	7.7050 ns/op	0.88
intersect array and set length 8	53.107 ns/op	90.206 ns/op	0.59
intersect bitArray bitLen 128	30.527 ns/op	32.921 ns/op	0.93
intersect array and set length 128	816.80 ns/op	1.0748 us/op	0.76
bitArray.getTrueBitIndexes() bitLen 128	2.3070 us/op	3.0360 us/op	0.76
bitArray.getTrueBitIndexes() bitLen 248	3.7440 us/op	5.0690 us/op	0.74
bitArray.getTrueBitIndexes() bitLen 512	8.7560 us/op	13.678 us/op	0.64
Buffer.concat 32 items	976.00 ns/op	1.3060 us/op	0.75
Uint8Array.set 32 items	1.4860 us/op	2.5370 us/op	0.59
Buffer.copy	1.6970 us/op	3.1780 us/op	0.53
Uint8Array.set - with subarray	2.6720 us/op	3.9240 us/op	0.68
Uint8Array.set - without subarray	1.4800 us/op	1.9910 us/op	0.74
getUint32 - dataview	278.00 ns/op	343.00 ns/op	0.81
getUint32 - manual	199.00 ns/op	284.00 ns/op	0.70
Set add up to 64 items then delete first	2.4481 us/op	3.1384 us/op	0.78
OrderedSet add up to 64 items then delete first	3.7671 us/op	5.4223 us/op	0.69
Set add up to 64 items then delete last	3.0224 us/op	4.2805 us/op	0.71
OrderedSet add up to 64 items then delete last	4.8935 us/op	5.1915 us/op	0.94
Set add up to 64 items then delete middle	2.9410 us/op	3.6497 us/op	0.81
OrderedSet add up to 64 items then delete middle	6.2278 us/op	7.9798 us/op	0.78
Set add up to 128 items then delete first	5.6411 us/op	8.0303 us/op	0.70
OrderedSet add up to 128 items then delete first	8.3291 us/op	15.008 us/op	0.55
Set add up to 128 items then delete last	5.2659 us/op	7.7626 us/op	0.68
OrderedSet add up to 128 items then delete last	8.7386 us/op	11.758 us/op	0.74
Set add up to 128 items then delete middle	5.1135 us/op	7.4560 us/op	0.69
OrderedSet add up to 128 items then delete middle	16.209 us/op	19.736 us/op	0.82
Set add up to 256 items then delete first	12.807 us/op	16.186 us/op	0.79
OrderedSet add up to 256 items then delete first	20.519 us/op	23.622 us/op	0.87
Set add up to 256 items then delete last	12.110 us/op	14.717 us/op	0.82
OrderedSet add up to 256 items then delete last	15.897 us/op	25.876 us/op	0.61
Set add up to 256 items then delete middle	11.240 us/op	15.320 us/op	0.73
OrderedSet add up to 256 items then delete middle	47.794 us/op	57.070 us/op	0.84
transfer serialized Status (84 B)	1.3960 us/op	1.7920 us/op	0.78
copy serialized Status (84 B)	1.2700 us/op	1.6600 us/op	0.77
transfer serialized SignedVoluntaryExit (112 B)	1.4710 us/op	1.8910 us/op	0.78
copy serialized SignedVoluntaryExit (112 B)	1.3310 us/op	1.6060 us/op	0.83
transfer serialized ProposerSlashing (416 B)	1.7430 us/op	2.9520 us/op	0.59
copy serialized ProposerSlashing (416 B)	1.9670 us/op	2.4450 us/op	0.80
transfer serialized Attestation (485 B)	2.2620 us/op	2.5250 us/op	0.90
copy serialized Attestation (485 B)	2.2390 us/op	2.3420 us/op	0.96
transfer serialized AttesterSlashing (33232 B)	2.1660 us/op	2.3800 us/op	0.91
copy serialized AttesterSlashing (33232 B)	7.1700 us/op	9.1950 us/op	0.78
transfer serialized Small SignedBeaconBlock (128000 B)	2.9950 us/op	3.5160 us/op	0.85
copy serialized Small SignedBeaconBlock (128000 B)	22.463 us/op	29.514 us/op	0.76
transfer serialized Avg SignedBeaconBlock (200000 B)	3.7990 us/op	4.1430 us/op	0.92
copy serialized Avg SignedBeaconBlock (200000 B)	34.630 us/op	42.536 us/op	0.81
transfer serialized BlobsSidecar (524380 B)	4.0900 us/op	4.6950 us/op	0.87
copy serialized BlobsSidecar (524380 B)	85.476 us/op	144.42 us/op	0.59
transfer serialized Big SignedBeaconBlock (1000000 B)	3.5080 us/op	5.6410 us/op	0.62
copy serialized Big SignedBeaconBlock (1000000 B)	201.63 us/op	267.15 us/op	0.75
pass gossip attestations to forkchoice per slot	2.9777 ms/op	3.3300 ms/op	0.89
forkChoice updateHead vc 100000 bc 64 eq 0	517.60 us/op	564.80 us/op	0.92
forkChoice updateHead vc 600000 bc 64 eq 0	3.1981 ms/op	4.2735 ms/op	0.75
forkChoice updateHead vc 1000000 bc 64 eq 0	5.6688 ms/op	6.7732 ms/op	0.84
forkChoice updateHead vc 600000 bc 320 eq 0	3.2532 ms/op	3.5246 ms/op	0.92
forkChoice updateHead vc 600000 bc 1200 eq 0	3.2433 ms/op	3.7537 ms/op	0.86
forkChoice updateHead vc 600000 bc 7200 eq 0	4.3036 ms/op	4.7980 ms/op	0.90
forkChoice updateHead vc 600000 bc 64 eq 1000	11.178 ms/op	12.947 ms/op	0.86
forkChoice updateHead vc 600000 bc 64 eq 10000	10.992 ms/op	12.071 ms/op	0.91
forkChoice updateHead vc 600000 bc 64 eq 300000	17.234 ms/op	17.849 ms/op	0.97
computeDeltas 500000 validators 300 proto nodes	4.0945 ms/op	5.1500 ms/op	0.80
computeDeltas 500000 validators 1200 proto nodes	4.1903 ms/op	4.7981 ms/op	0.87
computeDeltas 500000 validators 7200 proto nodes	4.1062 ms/op	4.9189 ms/op	0.83
computeDeltas 750000 validators 300 proto nodes	5.9916 ms/op	7.3694 ms/op	0.81
computeDeltas 750000 validators 1200 proto nodes	5.8951 ms/op	7.2950 ms/op	0.81
computeDeltas 750000 validators 7200 proto nodes	5.8338 ms/op	6.7431 ms/op	0.87
computeDeltas 1400000 validators 300 proto nodes	11.421 ms/op	12.514 ms/op	0.91
computeDeltas 1400000 validators 1200 proto nodes	11.293 ms/op	12.107 ms/op	0.93
computeDeltas 1400000 validators 7200 proto nodes	11.084 ms/op	11.524 ms/op	0.96
computeDeltas 2100000 validators 300 proto nodes	17.999 ms/op	18.069 ms/op	1.00
computeDeltas 2100000 validators 1200 proto nodes	16.755 ms/op	17.369 ms/op	0.96
computeDeltas 2100000 validators 7200 proto nodes	16.138 ms/op	17.746 ms/op	0.91
altair processAttestation - 250000 vs - 7PWei normalcase	2.6176 ms/op	2.2313 ms/op	1.17
altair processAttestation - 250000 vs - 7PWei worstcase	3.6269 ms/op	3.0812 ms/op	1.18
altair processAttestation - setStatus - 1/6 committees join	135.98 us/op	104.67 us/op	1.30
altair processAttestation - setStatus - 1/3 committees join	257.79 us/op	209.31 us/op	1.23
altair processAttestation - setStatus - 1/2 committees join	357.59 us/op	293.02 us/op	1.22
altair processAttestation - setStatus - 2/3 committees join	465.01 us/op	368.48 us/op	1.26
altair processAttestation - setStatus - 4/5 committees join	739.29 us/op	530.27 us/op	1.39
altair processAttestation - setStatus - 100% committees join	786.86 us/op	704.55 us/op	1.12
altair processBlock - 250000 vs - 7PWei normalcase	7.3656 ms/op	5.1921 ms/op	1.42
altair processBlock - 250000 vs - 7PWei normalcase hashState	32.934 ms/op	30.098 ms/op	1.09
altair processBlock - 250000 vs - 7PWei worstcase	47.740 ms/op	42.046 ms/op	1.14
altair processBlock - 250000 vs - 7PWei worstcase hashState	113.37 ms/op	92.813 ms/op	1.22
phase0 processBlock - 250000 vs - 7PWei normalcase	2.8880 ms/op	2.6002 ms/op	1.11
phase0 processBlock - 250000 vs - 7PWei worstcase	31.521 ms/op	27.396 ms/op	1.15
altair processEth1Data - 250000 vs - 7PWei normalcase	559.25 us/op	380.53 us/op	1.47
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15	12.755 us/op	7.5610 us/op	1.69
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219	61.161 us/op	40.297 us/op	1.52
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42	16.633 us/op	12.957 us/op	1.28
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18	11.693 us/op	8.3260 us/op	1.40
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020	208.21 us/op	139.59 us/op	1.49
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777	1.6317 ms/op	1.6916 ms/op	0.96
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384	1.9230 ms/op	2.8928 ms/op	0.66
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384	2.0619 ms/op	1.6729 ms/op	1.23
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384	5.1066 ms/op	3.6708 ms/op	1.39
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384	2.0763 ms/op	2.3037 ms/op	0.90
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384	5.7471 ms/op	3.8951 ms/op	1.48
Tree 40 250000 create	615.56 ms/op	234.63 ms/op	2.62
Tree 40 250000 get(125000)	198.45 ns/op	167.57 ns/op	1.18
Tree 40 250000 set(125000)	1.3585 us/op	799.54 ns/op	1.70
Tree 40 250000 toArray()	24.556 ms/op	20.598 ms/op	1.19
Tree 40 250000 iterate all - toArray() + loop	24.355 ms/op	22.045 ms/op	1.10
Tree 40 250000 iterate all - get(i)	68.246 ms/op	67.872 ms/op	1.01
Array 250000 create	5.2032 ms/op	3.9375 ms/op	1.32
Array 250000 clone - spread	5.3621 ms/op	1.6473 ms/op	3.26
Array 250000 get(125000)	0.50700 ns/op	0.44100 ns/op	1.15
Array 250000 set(125000)	0.54400 ns/op	0.46200 ns/op	1.18
Array 250000 iterate all - loop	115.26 us/op	104.11 us/op	1.11
phase0 afterProcessEpoch - 250000 vs - 7PWei	101.93 ms/op	99.668 ms/op	1.02
Array.fill - length 1000000	7.2186 ms/op	5.1340 ms/op	1.41
Array push - length 1000000	33.316 ms/op	19.208 ms/op	1.73
Array.get	0.46471 ns/op	0.32048 ns/op	1.45
Uint8Array.get	0.52783 ns/op	0.47165 ns/op	1.12
phase0 beforeProcessEpoch - 250000 vs - 7PWei	23.716 ms/op	19.317 ms/op	1.23
altair processEpoch - mainnet_e81889	409.20 ms/op	344.36 ms/op	1.19
mainnet_e81889 - altair beforeProcessEpoch	26.108 ms/op	20.025 ms/op	1.30
mainnet_e81889 - altair processJustificationAndFinalization	26.250 us/op	17.193 us/op	1.53
mainnet_e81889 - altair processInactivityUpdates	9.7705 ms/op	5.7576 ms/op	1.70
mainnet_e81889 - altair processRewardsAndPenalties	51.789 ms/op	55.128 ms/op	0.94
mainnet_e81889 - altair processRegistryUpdates	4.5080 us/op	3.7270 us/op	1.21
mainnet_e81889 - altair processSlashings	1.0900 us/op	591.00 ns/op	1.84
mainnet_e81889 - altair processEth1DataReset	1.1050 us/op	333.00 ns/op	3.32
mainnet_e81889 - altair processEffectiveBalanceUpdates	1.8516 ms/op	1.9343 ms/op	0.96
mainnet_e81889 - altair processSlashingsReset	11.693 us/op	5.0700 us/op	2.31
mainnet_e81889 - altair processRandaoMixesReset	11.481 us/op	7.3920 us/op	1.55
mainnet_e81889 - altair processHistoricalRootsUpdate	1.0350 us/op	793.00 ns/op	1.31
mainnet_e81889 - altair processParticipationFlagUpdates	4.0470 us/op	1.9700 us/op	2.05
mainnet_e81889 - altair processSyncCommitteeUpdates	966.00 ns/op	693.00 ns/op	1.39
mainnet_e81889 - altair afterProcessEpoch	102.43 ms/op	101.90 ms/op	1.01
capella processEpoch - mainnet_e217614	1.4049 s/op	1.2677 s/op	1.11
mainnet_e217614 - capella beforeProcessEpoch	76.210 ms/op	69.011 ms/op	1.10
mainnet_e217614 - capella processJustificationAndFinalization	19.051 us/op	22.114 us/op	0.86
mainnet_e217614 - capella processInactivityUpdates	19.200 ms/op	19.118 ms/op	1.00
mainnet_e217614 - capella processRewardsAndPenalties	261.73 ms/op	269.55 ms/op	0.97
mainnet_e217614 - capella processRegistryUpdates	12.004 us/op	16.006 us/op	0.75
mainnet_e217614 - capella processSlashings	381.00 ns/op	416.00 ns/op	0.92
mainnet_e217614 - capella processEth1DataReset	312.00 ns/op	318.00 ns/op	0.98
mainnet_e217614 - capella processEffectiveBalanceUpdates	12.975 ms/op	13.756 ms/op	0.94
mainnet_e217614 - capella processSlashingsReset	2.8330 us/op	3.1480 us/op	0.90
mainnet_e217614 - capella processRandaoMixesReset	4.5880 us/op	7.0780 us/op	0.65
mainnet_e217614 - capella processHistoricalRootsUpdate	355.00 ns/op	507.00 ns/op	0.70
mainnet_e217614 - capella processParticipationFlagUpdates	3.4800 us/op	1.8200 us/op	1.91
mainnet_e217614 - capella afterProcessEpoch	243.01 ms/op	247.11 ms/op	0.98
phase0 processEpoch - mainnet_e58758	424.69 ms/op	460.89 ms/op	0.92
mainnet_e58758 - phase0 beforeProcessEpoch	91.352 ms/op	98.938 ms/op	0.92
mainnet_e58758 - phase0 processJustificationAndFinalization	15.817 us/op	20.880 us/op	0.76
mainnet_e58758 - phase0 processRewardsAndPenalties	28.210 ms/op	33.745 ms/op	0.84
mainnet_e58758 - phase0 processRegistryUpdates	8.7880 us/op	7.8900 us/op	1.11
mainnet_e58758 - phase0 processSlashings	457.00 ns/op	425.00 ns/op	1.08
mainnet_e58758 - phase0 processEth1DataReset	628.00 ns/op	556.00 ns/op	1.13
mainnet_e58758 - phase0 processEffectiveBalanceUpdates	1.4465 ms/op	1.5574 ms/op	0.93
mainnet_e58758 - phase0 processSlashingsReset	4.3280 us/op	3.0490 us/op	1.42
mainnet_e58758 - phase0 processRandaoMixesReset	5.1780 us/op	4.9660 us/op	1.04
mainnet_e58758 - phase0 processHistoricalRootsUpdate	653.00 ns/op	693.00 ns/op	0.94
mainnet_e58758 - phase0 processParticipationRecordUpdates	5.7720 us/op	6.1460 us/op	0.94
mainnet_e58758 - phase0 afterProcessEpoch	83.611 ms/op	93.483 ms/op	0.89
phase0 processEffectiveBalanceUpdates - 250000 normalcase	2.3941 ms/op	2.3220 ms/op	1.03
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5	2.2650 ms/op	2.7812 ms/op	0.81
altair processInactivityUpdates - 250000 normalcase	22.336 ms/op	20.484 ms/op	1.09
altair processInactivityUpdates - 250000 worstcase	19.478 ms/op	19.346 ms/op	1.01
phase0 processRegistryUpdates - 250000 normalcase	8.3520 us/op	10.546 us/op	0.79
phase0 processRegistryUpdates - 250000 badcase_full_deposits	360.51 us/op	366.53 us/op	0.98
phase0 processRegistryUpdates - 250000 worstcase 0.5	139.65 ms/op	144.62 ms/op	0.97
altair processRewardsAndPenalties - 250000 normalcase	50.888 ms/op	44.294 ms/op	1.15
altair processRewardsAndPenalties - 250000 worstcase	44.206 ms/op	46.858 ms/op	0.94
phase0 getAttestationDeltas - 250000 normalcase	11.192 ms/op	11.960 ms/op	0.94
phase0 getAttestationDeltas - 250000 worstcase	9.7479 ms/op	12.489 ms/op	0.78
phase0 processSlashings - 250000 worstcase	122.68 us/op	110.96 us/op	1.11
altair processSyncCommitteeUpdates - 250000	140.69 ms/op	154.29 ms/op	0.91
BeaconState.hashTreeRoot - No change	271.00 ns/op	448.00 ns/op	0.60
BeaconState.hashTreeRoot - 1 full validator	121.63 us/op	135.63 us/op	0.90
BeaconState.hashTreeRoot - 32 full validator	1.2614 ms/op	1.2190 ms/op	1.03
BeaconState.hashTreeRoot - 512 full validator	15.358 ms/op	16.398 ms/op	0.94
BeaconState.hashTreeRoot - 1 validator.effectiveBalance	179.43 us/op	163.47 us/op	1.10
BeaconState.hashTreeRoot - 32 validator.effectiveBalance	2.3053 ms/op	3.0442 ms/op	0.76
BeaconState.hashTreeRoot - 512 validator.effectiveBalance	29.205 ms/op	46.331 ms/op	0.63
BeaconState.hashTreeRoot - 1 balances	117.54 us/op	187.56 us/op	0.63
BeaconState.hashTreeRoot - 32 balances	887.16 us/op	1.4102 ms/op	0.63
BeaconState.hashTreeRoot - 512 balances	9.4899 ms/op	13.905 ms/op	0.68
BeaconState.hashTreeRoot - 250000 balances	187.11 ms/op	222.11 ms/op	0.84
aggregationBits - 2048 els - zipIndexesInBitList	35.243 us/op	36.180 us/op	0.97
byteArrayEquals 32	57.449 ns/op	62.570 ns/op	0.92
Buffer.compare 32	17.995 ns/op	19.089 ns/op	0.94
byteArrayEquals 1024	1.7098 us/op	1.7836 us/op	0.96
Buffer.compare 1024	27.674 ns/op	28.090 ns/op	0.99
byteArrayEquals 16384	26.224 us/op	27.260 us/op	0.96
Buffer.compare 16384	210.75 ns/op	220.47 ns/op	0.96
byteArrayEquals 123687377	197.09 ms/op	218.39 ms/op	0.90
Buffer.compare 123687377	8.8053 ms/op	10.260 ms/op	0.86
byteArrayEquals 32 - diff last byte	53.509 ns/op	58.625 ns/op	0.91
Buffer.compare 32 - diff last byte	17.604 ns/op	20.344 ns/op	0.87
byteArrayEquals 1024 - diff last byte	1.6119 us/op	1.7945 us/op	0.90
Buffer.compare 1024 - diff last byte	26.408 ns/op	30.815 ns/op	0.86
byteArrayEquals 16384 - diff last byte	25.682 us/op	28.208 us/op	0.91
Buffer.compare 16384 - diff last byte	207.81 ns/op	203.70 ns/op	1.02
byteArrayEquals 123687377 - diff last byte	195.35 ms/op	208.00 ms/op	0.94
Buffer.compare 123687377 - diff last byte	8.7436 ms/op	9.6462 ms/op	0.91
byteArrayEquals 32 - random bytes	5.3760 ns/op	6.3780 ns/op	0.84
Buffer.compare 32 - random bytes	17.510 ns/op	19.543 ns/op	0.90
byteArrayEquals 1024 - random bytes	6.7940 ns/op	5.5610 ns/op	1.22
Buffer.compare 1024 - random bytes	17.414 ns/op	18.745 ns/op	0.93
byteArrayEquals 16384 - random bytes	5.3040 ns/op	5.5350 ns/op	0.96
Buffer.compare 16384 - random bytes	17.443 ns/op	18.241 ns/op	0.96
byteArrayEquals 123687377 - random bytes	6.7500 ns/op	6.8700 ns/op	0.98
Buffer.compare 123687377 - random bytes	19.220 ns/op	20.230 ns/op	0.95
regular array get 100000 times	40.840 us/op	40.928 us/op	1.00
wrappedArray get 100000 times	33.749 us/op	37.994 us/op	0.89
arrayWithProxy get 100000 times	13.712 ms/op	14.464 ms/op	0.95
ssz.Root.equals	47.199 ns/op	51.942 ns/op	0.91
byteArrayEquals	46.760 ns/op	49.604 ns/op	0.94
Buffer.compare	10.675 ns/op	12.653 ns/op	0.84
shuffle list - 16384 els	6.4760 ms/op	8.2827 ms/op	0.78
shuffle list - 250000 els	96.847 ms/op	109.83 ms/op	0.88
processSlot - 1 slots	12.775 us/op	19.059 us/op	0.67
processSlot - 32 slots	3.8765 ms/op	3.5220 ms/op	1.10
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei	40.456 ms/op	42.796 ms/op	0.95
getCommitteeAssignments - req 1 vs - 250000 vc	2.1802 ms/op	2.6644 ms/op	0.82
getCommitteeAssignments - req 100 vs - 250000 vc	4.2645 ms/op	5.0722 ms/op	0.84
getCommitteeAssignments - req 1000 vs - 250000 vc	4.6105 ms/op	5.3373 ms/op	0.86
findModifiedValidators - 10000 modified validators	324.05 ms/op	268.23 ms/op	1.21
findModifiedValidators - 1000 modified validators	212.59 ms/op	176.98 ms/op	1.20
findModifiedValidators - 100 modified validators	205.69 ms/op	190.44 ms/op	1.08
findModifiedValidators - 10 modified validators	162.51 ms/op	163.37 ms/op	0.99
findModifiedValidators - 1 modified validators	157.86 ms/op	177.11 ms/op	0.89
findModifiedValidators - no difference	150.35 ms/op	248.14 ms/op	0.61
compare ViewDUs	3.5832 s/op	3.4711 s/op	1.03
compare each validator Uint8Array	1.8038 s/op	1.3499 s/op	1.34
compare ViewDU to Uint8Array	1.6303 s/op	1.3222 s/op	1.23
migrate state 1000000 validators, 24 modified, 0 new	1.2062 s/op	670.74 ms/op	1.80
migrate state 1000000 validators, 1700 modified, 1000 new	1.1179 s/op	954.61 ms/op	1.17
migrate state 1000000 validators, 3400 modified, 2000 new	1.3184 s/op	1.1845 s/op	1.11
migrate state 1500000 validators, 24 modified, 0 new	957.42 ms/op	787.90 ms/op	1.22
migrate state 1500000 validators, 1700 modified, 1000 new	1.1815 s/op	1.0710 s/op	1.10
migrate state 1500000 validators, 3400 modified, 2000 new	1.3737 s/op	1.4247 s/op	0.96
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei	5.0200 ns/op	6.2900 ns/op	0.80
state getBlockRootAtSlot - 250000 vs - 7PWei	634.57 ns/op	1.2018 us/op	0.53
computeProposers - vc 250000	8.3860 ms/op	11.480 ms/op	0.73
computeEpochShuffling - vc 250000	97.356 ms/op	113.61 ms/op	0.86
getNextSyncCommittee - vc 250000	124.24 ms/op	193.74 ms/op	0.64
computeSigningRoot for AttestationData	24.113 us/op	33.125 us/op	0.73
hash AttestationData serialized data then Buffer.toString(base64)	1.6364 us/op	2.3510 us/op	0.70
toHexString serialized data	909.90 ns/op	1.9590 us/op	0.46
Buffer.toString(base64)	209.74 ns/op	302.29 ns/op	0.69
nodejs block root to RootHex using toHex	164.20 ns/op	242.33 ns/op	0.68
nodejs block root to RootHex using toRootHex	96.892 ns/op	151.37 ns/op	0.64
browser block root to RootHex using the deprecated toHexString	256.01 ns/op	473.43 ns/op	0.54
browser block root to RootHex using toHex	191.78 ns/op	394.34 ns/op	0.49
browser block root to RootHex using toRootHex	173.48 ns/op	202.64 ns/op	0.86

by benchmarkbot/action

…Cache

This reverts commit 104aa56.

packages/state-transition/src/cache/epochTransitionCache.ts

packages/state-transition/src/cache/epochCache.ts

…o Uint32Array

matthewkeil · 2024-09-06T03:15:43Z

Epoch Transition Metrics

Unstable to Feat2 comparison (`holesky lg1k`)

Epoch Transition Heatmap

Mainet

`stable-mainet`

`unstable-mainet`

`feat2-mainet`

Holesky `lg1k`

`stable-holesky-lg1k`

`unstable-holesky-lg1k`

`feat2-holesky-lg1k`

Holesky `sm1v`

`stable-holesky-sm1v`

`unstable-holesky-sm1v`

`feat2-holesky-sm1v`

`feat2-mainet`

Here is the 4day metrics comparing the branch on this host vs the shuffling changes. While not a direct comparison to unstable its still representative

packages/state-transition/src/cache/epochCache.ts

wemeetagain

This looks good

packages/state-transition/src/cache/epochTransitionCache.ts

twoeths

looks awesome, I think we should wait for v1.22 to be released first before merging this

nflaig

is there a reason why we use both upcoming and next epoch terminology?

packages/beacon-node/src/api/impl/beacon/state/index.ts

packages/light-client/test/unit/webEsmBundle.browser.test.ts

packages/state-transition/src/cache/epochCache.ts

…te PR

matthewkeil · 2024-09-19T08:10:02Z

is there a reason why we use both upcoming and next epoch terminology?

Yes. The slot gets incremented between processEpoch and afterProcessEpoch so technically the epoch has incremented there and its confusing in some contexts that the "next" epoch is actually the "current" epoch so to standardize terminology across the epoch transition phases I think using "upcoming" epoch for the one directly after the state transition is less confusing. The "next" fields like nextProposer during the transition are actually for epoch n+2 (epochAfterUpcoming) which is confusing because we update the "epochCtx.nextEpoch" right at the end of the epoch transition so "next" is referring to two different epochs.

wemeetagain · 2024-09-19T10:12:23Z

Yes. ...

Can you commit this somewhere in the code as a jsdoc comment?

nflaig · 2024-09-19T11:47:28Z

packages/beacon-node/src/api/impl/validator/index.ts

+      const shuffling = await chain.shufflingCache.get(epoch, decisionRoot);
+      if (!shuffling) {
+        throw new ApiError(
+          400,


should be aligned to the other error

twoeths reviewed Jul 10, 2024

View reviewed changes

packages/state-transition/src/cache/epochCache.ts Show resolved Hide resolved

twoeths reviewed Jul 10, 2024

View reviewed changes

packages/state-transition/src/cache/epochCache.ts Outdated Show resolved Hide resolved

twoeths reviewed Jul 10, 2024

View reviewed changes

packages/state-transition/src/cache/epochCache.ts Outdated Show resolved Hide resolved

matthewkeil and others added 25 commits August 11, 2024 23:25

feat: add ShufflingCache to EpochCache

46b6b3f

fix: implementation in state-transition for EpochCache with Shuffling…

99269ce

…Cache

feat: remove shufflingCache.processState

52187c9

feat: implement ShufflingCache changes in beacon-node

b6b2f20

feat: pass shufflingCache when loading cached state from db

4110dc0

test: fix state-transition tests for EpochCache changes

04e4a7b

feat: Pass shufflingCache to EpochCache at startup

ff2d520

test: fix slot off by one for decision root in perf test

53ed72e

chore: use ?. syntax

8faee3f

chore: refactoring

d69c8b1

feat: add comments and clean up afterProcessEpoch

cfa97ad

fix: perf test slot incrementing

78a60a2

fix: remove MockShufflingCache

9b0ffe2

Revert "chore: refactoring"

13f5996

This reverts commit 104aa56.

refactor: shufflingCache getters

d8d69c9

refactor: shufflingCache setters

89798a5

refactor: build and getOrBuild

81449e4

docs: add comments to ShufflingCache methods

5799676

chore: lint issues

9a0ca70

test: update tests in beacon-node

10dd105

chore: lint

bb3fd1c

feat: get shufflings from cache for API

59f6d3f

feat: minTimeDelayToBuildShuffling cli flag

bae40cc

test: fix shufflingCache promise insertion test

b8269ef

fix: rebase conflicts

a0e7e60

matthewkeil added 3 commits August 28, 2024 03:05

fix: touch up metrics per PR comments

1ab811c

Merge branch 'unstable' into mkeil/shuffling-refactor

0e8eeb7

fix: merge conflict

fd8196e

matthewkeil force-pushed the mkeil/shuffling-refactor branch from 7d24ec9 to fd8196e Compare September 3, 2024 08:57

chore: lint

4f777c8

matthewkeil requested a review from twoeths September 3, 2024 10:09

twoeths reviewed Sep 5, 2024

View reviewed changes

packages/state-transition/src/cache/epochTransitionCache.ts Outdated Show resolved Hide resolved

twoeths reviewed Sep 5, 2024

View reviewed changes

packages/state-transition/src/cache/epochCache.ts Outdated Show resolved Hide resolved

matthewkeil added 2 commits September 5, 2024 22:44

refactor: add scope around activeIndices to GC arrays

222908c

feat: directly use Uint32Array instead of transcribing number array t…

ffbbfea

…o Uint32Array

twoeths reviewed Sep 6, 2024

View reviewed changes

packages/state-transition/src/cache/epochCache.ts Outdated Show resolved Hide resolved

refactor: activeIndices per tuyen comment

7f6d420

matthewkeil requested a review from twoeths September 6, 2024 04:52

wemeetagain reviewed Sep 9, 2024

View reviewed changes

packages/state-transition/src/cache/epochTransitionCache.ts Outdated Show resolved Hide resolved

refactor: rename to epochAfterNext

afc03f2

matthewkeil requested a review from wemeetagain September 9, 2024 23:31

twoeths mentioned this pull request Sep 14, 2024

feat: implement BalancesTreeCache #7084

Merged

chore: review PR

d1d0e22

twoeths previously approved these changes Sep 16, 2024

View reviewed changes

matthewkeil added the status-do-not-merge Merging this issue will break the build. Do not merge! label Sep 16, 2024

nflaig reviewed Sep 16, 2024

View reviewed changes

packages/beacon-node/src/api/impl/beacon/state/index.ts Outdated Show resolved Hide resolved

packages/light-client/test/unit/webEsmBundle.browser.test.ts Outdated Show resolved Hide resolved

packages/state-transition/src/cache/epochCache.ts Show resolved Hide resolved

matthewkeil added 2 commits September 19, 2024 14:56

feat: update no shuffling ApiError to 500 status

c5c5772

fix: add back unnecessary eslint directive. to be remove under separa…

88d3105

…te PR

matthewkeil dismissed twoeths’s stale review via 88d3105 September 19, 2024 08:05

matthewkeil requested review from nflaig and twoeths September 19, 2024 08:10

nflaig reviewed Sep 19, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: async shuffling refactor #6938

feat: async shuffling refactor #6938

matthewkeil commented Jul 8, 2024 •

edited

Loading

codecov bot commented Jul 9, 2024 •

edited

Loading

github-actions bot commented Jul 11, 2024 •

edited

Loading

matthewkeil commented Sep 6, 2024 •

edited

Loading

wemeetagain left a comment

twoeths left a comment

nflaig left a comment

matthewkeil commented Sep 19, 2024

wemeetagain commented Sep 19, 2024

nflaig Sep 19, 2024

feat: async shuffling refactor #6938

Are you sure you want to change the base?

feat: async shuffling refactor #6938

Conversation

matthewkeil commented Jul 8, 2024 • edited Loading

codecov bot commented Jul 9, 2024 • edited Loading

Codecov Report

github-actions bot commented Jul 11, 2024 • edited Loading

Performance Report

matthewkeil commented Sep 6, 2024 • edited Loading

Epoch Transition Metrics

Unstable to Feat2 comparison (holesky lg1k)

Epoch Transition Heatmap

Mainet

stable-mainet

unstable-mainet

feat2-mainet

Holesky lg1k

stable-holesky-lg1k

unstable-holesky-lg1k

feat2-holesky-lg1k

Holesky sm1v

stable-holesky-sm1v

unstable-holesky-sm1v

feat2-holesky-sm1v

feat2-mainet

wemeetagain left a comment

Choose a reason for hiding this comment

twoeths left a comment

Choose a reason for hiding this comment

nflaig left a comment

Choose a reason for hiding this comment

matthewkeil commented Sep 19, 2024

wemeetagain commented Sep 19, 2024

nflaig Sep 19, 2024

Choose a reason for hiding this comment

matthewkeil commented Jul 8, 2024 •

edited

Loading

codecov bot commented Jul 9, 2024 •

edited

Loading

github-actions bot commented Jul 11, 2024 •

edited

Loading

matthewkeil commented Sep 6, 2024 •

edited

Loading

Unstable to Feat2 comparison (`holesky lg1k`)

`stable-mainet`

`unstable-mainet`

`feat2-mainet`

Holesky `lg1k`

`stable-holesky-lg1k`

`unstable-holesky-lg1k`

`feat2-holesky-lg1k`

Holesky `sm1v`

`stable-holesky-sm1v`

`unstable-holesky-sm1v`

`feat2-holesky-sm1v`

`feat2-mainet`