-
Notifications
You must be signed in to change notification settings - Fork 573
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SI Crash under load in FranzGoVerifiableWithSiTest
.test_si_with_timeboxed
,test_si_without_timeboxed
#5613
Comments
FranzGoVerifiableWithSiTest
.test_si_with_timeboxed
FranzGoVerifiableWithSiTest
.test_si_with_timeboxed
Weee I think I did it. IMPORTANT: this was decoded based on matching sha1 not based on time (ie cdt chooses the latest build not a specific build). So it is possible that this is wrong. I will double check.
|
I have a CI failure with a different test that shows basically the same issue with
|
if I read that correctly the crash happens when abseil doesn't specify what |
I haven't looked too deeply so maybe the suggestion doesn't make sense, but it also looks like we're susceptible to iterator invalidation when inserting and deleting from the map (see here), and it looks like |
@andrwng good catch! I see that So to reproduce this issue we need to run |
I was able to reproduce the crash via inserting Ideas how to fix it:
@Lazin WDYT? |
|
instead of using |
FranzGoVerifiableWithSiTest
.test_si_with_timeboxed
FranzGoVerifiableWithSiTest
.test_si_with_timeboxed
,test_si_without_timeboxed
This was a clustered ducktape run from 20th July https://buildkite.com/redpanda/vtools/builds/2906
Tail of crashed node (
ip-172-31-54-196
) log is:These clustered ducktape runs use nightly packages, so it should be possible to download the packages from that night (including debug symbols package) and decode that backtrace.
The text was updated successfully, but these errors were encountered: