[Config Change] Stylus LRU cache with bytes capacity instead of number of entries capacity #2595

diegoximenes · 2024-08-20T17:58:05Z

Stylus LRU cache with bytes capacity instead of number of entries capacity.
It infers the size of each entry by computing the number of bytes related to the serialized wasm module.

This PR also removes -Wall from cgo CFLAGS.
go doesn't handle it well: https://stackoverflow.com/questions/56633817/warning-unused-variable-cgo-a.
With it the following warning was being generated without need:

cgo-gcc-prolog: In function ‘_cgo_80afc8d9354c_Cfunc_stylus_clear_lru_cache’:
cgo-gcc-prolog:139:49: warning: unused variable ‘_cgo_a’ [-Wunused-variable]

diegoximenes · 2024-08-22T13:13:25Z

execution/gethexec/blockchain.go

@@ -75,7 +75,7 @@ var DefaultCachingConfig = CachingConfig{
 	SnapshotRestoreGasLimit:            300_000_000_000,
 	MaxNumberOfBlocksToSkipStateSaving: 0,
 	MaxAmountOfGasToSkipStateSaving:    0,
-	StylusLRUCache:                     256,
+	StylusLRUCacheCapacity:             256,


Suggestions on the default value?

I am fine with 256 for now. If we frequently run out, we can change it in a later PR.

tsahee

realised I'm sitting on these Pending comments for a while .. sorry

tsahee · 2024-08-23T18:47:42Z

arbitrator/stylus/src/cache.rs

+    fn weight(&self, _key: &CacheKey, val: &CacheItem) -> usize {
+        // clru defines that each entry consumes (weight + 1) of the cache capacity.
+        // We subtract 1 since we only want to use the weight as the size of the entry.
+        val.asm_size_estimate_bytes.saturating_sub(1)


Actually - a more exact number would be something like.. asm_size_estimate + 64/128
Since you are counting bytes, and every entry in the cache has overhead of the cache-key, cacheItem and some internal pointers. Exact number is not too important.

Added the 128 bytes to account for overheads that you mentioned :).

I kept taking into account that clru defines that each entry consumes (weight + 1) of the cache capacity though.
This simplifies tests that checks the expected cache size.

arbitrator/stylus/src/cache.rs

tsahee · 2024-08-23T20:42:53Z

arbitrator/stylus/src/cache.rs

+        let count = cache.lru.len();
+        let metrics = LruCacheMetrics {
+            // add 1 to each entry to account that we subtracted 1 in the weight calculation
+            size_bytes: (cache.lru.weight() + count).try_into().unwrap(),


Normal entry is order of magnitude of 100,000. a few bytes per entry isn't worth fussing over (I'd go with weight = deser_size + 128 and return lru.weight() from here)

Added the 128 bytes to account for overheads that you mentioned :).

I kept taking into account that clru defines that each entry consumes (weight + 1) of the cache capacity though.
This simplifies tests that checks the expected cache size.

tsahee · 2024-08-28T00:12:31Z

arbitrator/stylus/src/lib.rs

+/// Clears lru cache.
+/// Only used for testing purposes.
+#[no_mangle]
+pub extern "C" fn stylus_clear_lru_cache() {


wont stylus_set_cache_lru_capacity(0) and then resize-again work?

Nope :(, capacity must be non zero https://docs.rs/clru/0.6.2/clru/struct.CLruCache.html#method.resize

tsahee · 2024-08-28T01:17:57Z

arbos/programs/native.go

@@ -7,7 +7,7 @@
 package programs

 /*
-#cgo CFLAGS: -g -Wall -I../../target/include/
+#cgo CFLAGS: -g -I../../target/include/


go doesn't handle it well: https://stackoverflow.com/questions/56633817/warning-unused-variable-cgo-a.
With it the following warning was being generated without need:

cgo-gcc-prolog: In function ‘_cgo_80afc8d9354c_Cfunc_stylus_clear_lru_cache’: cgo-gcc-prolog:139:49: warning: unused variable ‘_cgo_a’ [-Wunused-variable]

execution/gethexec/blockchain.go

eljobe · 2024-08-29T08:06:21Z

execution/gethexec/blockchain.go

@@ -75,7 +75,7 @@ var DefaultCachingConfig = CachingConfig{
 	SnapshotRestoreGasLimit:            300_000_000_000,
 	MaxNumberOfBlocksToSkipStateSaving: 0,
 	MaxAmountOfGasToSkipStateSaving:    0,
-	StylusLRUCache:                     256,
+	StylusLRUCacheCapacity:             256,


I am fine with 256 for now. If we frequently run out, we can change it in a later PR.

execution/gethexec/executionengine.go

eljobe

LGTM

gligneul

LGTM

arbos/programs/native.go

arbitrator/stylus/src/cache.rs

…us LRU cache entry

…trics

cla-bot bot added the s Automatically added by the CLA bot if the creator of a PR is registered as having signed the CLA. label Aug 20, 2024

diegoximenes mentioned this pull request Aug 20, 2024

AsmSizeEstimateKb in CacheWasmRust OffchainLabs/go-ethereum#351

Closed

diegoximenes force-pushed the stylus_cache_bytes_capacity branch 3 times, most recently from 2961c11 to b90c32a Compare August 22, 2024 13:08

diegoximenes commented Aug 22, 2024

View reviewed changes

diegoximenes changed the title ~~Stylus LRU cache with kilobytes capacity instead of number of entries capacity~~ [Config Change] Stylus LRU cache with kilobytes capacity instead of number of entries capacity Aug 22, 2024

diegoximenes marked this pull request as ready for review August 22, 2024 14:58

diegoximenes changed the title ~~[Config Change] Stylus LRU cache with kilobytes capacity instead of number of entries capacity~~ [Config Change] Stylus LRU cache with bytes capacity instead of number of entries capacity Aug 22, 2024

diegoximenes requested review from eljobe, gligneul and magicxyyz August 22, 2024 14:58

diegoximenes force-pushed the stylus_cache_bytes_capacity branch from e91655c to 165d484 Compare August 27, 2024 13:31

tsahee reviewed Aug 27, 2024

View reviewed changes

tsahee reviewed Aug 28, 2024

View reviewed changes

diegoximenes requested a review from tsahee August 28, 2024 20:49

eljobe previously approved these changes Aug 29, 2024

View reviewed changes

diegoximenes dismissed eljobe’s stale review via b514e73 August 29, 2024 13:30

diegoximenes requested a review from eljobe August 29, 2024 13:32

eljobe previously approved these changes Aug 30, 2024

View reviewed changes

gligneul previously approved these changes Sep 5, 2024

View reviewed changes

arbos/programs/native.go Show resolved Hide resolved

arbitrator/stylus/src/cache.rs Show resolved Hide resolved

diegoximenes added 8 commits September 6, 2024 10:17

First step in lru to lru_mem migration in stylus cache

fab821e

stylus-lru-cache-size instead of stylus-lru-cache flag

e4c9406

Uses asm size estimate to define amount of heap memory used by a styl…

a3b2dd7

…us LRU cache entry

Basic stylus cache metrics

7e0de89

clru instead of lru-mem

9bc34de

Simplifies stylus lru cache metrics retrieval

f4c723c

AsmSizeEstimate to AsmSizeEstimateKb, CLI rust cache size flag in Mb

098fca5

Fixes weight of stylus cache entry

6261168

diegoximenes added 27 commits September 6, 2024 10:49

Fixes testcompile.go

4ac5698

Adjusts default stylus lru cache capacity

43e0bf6

Updates comment in stylus cache weight computation

516eceb

Rename asm_size_estimate to asm_size_estimate_kb

0bf9191

Improves comments on why to empty counters in get_lru_metrics call

38691b1

Rename asm_size_estimate to asm_size_estimate_kb

42631ea

Adds comment on periodic procedure that retrieves stylus lru cache me…

fa9a932

…trics

Updates comment TestWasmLruCache

d9bf584

Renames StylusLRUCacheSizeMb to StylusLRUCacheSize

ce46676

Updates lru cache metrics data sizes

7cb4fa1

Simplifies weight computation by using saturating_sub

21217f1

Uses if let instead of match

7e6b759

Removes unused return

fe2377c

Creates a DOES_NOT_FIT_MSG static

e439bc8

Uses is_err instead of let Err(_)

281f608

Format rs files

a2f1b60

Relies on module.serialize to estimate asm size

761f84a

Renames resize to set capacity in stylus cache

21664e9

Don't hardcode wasm sizes in TestWasmLruCache

b7a949c

Removes unnecessary calls to ExecutionEngine regarding Wasm LRU cache

bbed30a

Removes unsafe from stylus_get_asm_size_estimate_bytes

17a62da

Format rs files

8a3d6ef

From println to eprintln in stylus cache

eaa04db

Renames asm_size_estimate_bytes to lru_entry_size_estimate_bytes

1449a7b

Adds extra bytes to lru_entry_size_bytes to account for overheads

ba9e9d1

Format rs files

5a82f00

Renames rustCacheCapacityMb to rustCacheCapacityMB

f144eb1

diegoximenes dismissed stale reviews from gligneul and eljobe via f144eb1 September 6, 2024 18:39

diegoximenes force-pushed the stylus_cache_bytes_capacity branch from b514e73 to f144eb1 Compare September 6, 2024 18:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Config Change] Stylus LRU cache with bytes capacity instead of number of entries capacity #2595

[Config Change] Stylus LRU cache with bytes capacity instead of number of entries capacity #2595

diegoximenes commented Aug 20, 2024 •

edited

Loading

diegoximenes Aug 22, 2024

eljobe Aug 29, 2024

tsahee left a comment

tsahee Aug 23, 2024

diegoximenes Aug 28, 2024

tsahee Aug 23, 2024

diegoximenes Aug 28, 2024

tsahee Aug 28, 2024

diegoximenes Aug 28, 2024

tsahee Aug 28, 2024

diegoximenes Aug 28, 2024

eljobe Aug 29, 2024

eljobe left a comment

gligneul left a comment

[Config Change] Stylus LRU cache with bytes capacity instead of number of entries capacity #2595

Are you sure you want to change the base?

[Config Change] Stylus LRU cache with bytes capacity instead of number of entries capacity #2595

Conversation

diegoximenes commented Aug 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tsahee left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eljobe left a comment

Choose a reason for hiding this comment

gligneul left a comment

Choose a reason for hiding this comment

diegoximenes commented Aug 20, 2024 •

edited

Loading