[Networking] Fixing flakey TestGossipSubSpamMitigationIntegration #5095
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes the flakey behavior with
TestGossipSubSpamMitigationIntegration
.Test scenario:
The test assesses the comprehensive effectiveness of our GossipSub control message spam mitigation system. It initiates with a spammer, representing an attacker, bombarding a victim node through various control message spam attacks. The expectation is for the victim's RPC inspector module to detect these attacks and report them to the GossipSub score registry. The registry, in turn, penalizes the spammer based on the GossipSub's application-specific score. The severity of the attack is calibrated to be high, ensuring that the spammer's penalty eventually crosses the graylisting threshold. This results in the suppression of all GossipSub RPC exchanges between the spammer and the victim.
Flakiness root cause:
At the test's conclusion, we scrutinize the GossipSub RPC interactions between spammer and victim, focusing on the absence of pubsub message exchange. This absence suggests the victim has penalized the spammer. However, previously, the test hurriedly assessed the pubsub exchange capability without confirming if the penalty was substantial enough for graylisting. Given the 5-second timeout to verify pubsub exchange disruption, the spammer's penalty could diminish to a level where it regains pubsub exchange capabilities with the victim. This could lead to inconsistent test outcomes.
Fix implemented in this PR:
To address the issue, this PR implements two key changes:
To confirm the test's resolution, we execute it in four sets, each consisting of 20 consecutive runs, implementing a fail-fast approach. This means if any single execution in a set fails, the whole set is terminated. Success is determined by the test completing all 80 runs without failure.