ducky: add si test with blocked s3 #4478
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
firewall_blocked will be used also for shadow indexing test
Lazin
reviewed
Apr 28, 2022
NyaliaLui
suggested changes
Apr 28, 2022
dotnwat
previously requested changes
Apr 29, 2022
There was a problem hiding this comment.
looks like we're seeing one failure of one of the new tests
test_id: rptest.tests.shadow_indexing_firewall_test.ShadowIndexingFirewallTest.test_consume_from_blocked_s3
--
| status: FAIL
| run time: 2 minutes 38.924 seconds
|
|
| TimeoutError('Segments were not created')
| Traceback (most recent call last):
| File "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 135, in run
| data = self.run_test()
| File "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 227, in run_test
| return self.test_context.function(self.test)
| File "/root/tests/rptest/services/cluster.py", line 35, in wrapped
| r = f(self, *args, **kwargs)
| File "/root/tests/rptest/tests/shadow_indexing_firewall_test.py", line 73, in test_consume_from_blocked_s3
| produce_until_segments(redpanda=self.redpanda,
| File "/root/tests/rptest/util.py", line 112, in produce_until_segments
| wait_until(done,
| File "/usr/local/lib/python3.9/dist-packages/ducktape/utils/util.py", line 58, in wait_until
| raise TimeoutError(err_msg() if callable(err_msg) else err_msg) from last_exception
| ducktape.errors.TimeoutError: Segments were not created
hmm haven't seen that on CDT, but I guess enabling deletion after segments are produced should help, will fix it |
|
CI Failure not connected with new test: #4489 |
jcsp
reviewed
Apr 29, 2022
| si_settings = SISettings(cloud_storage_reconciliation_interval_ms=500, | ||
| cloud_storage_max_connections=5, | ||
| log_segment_size=self.log_segment_size) | ||
| self.s3_bucket_name = si_settings.cloud_storage_bucket |
| self.kafka_tools = KafkaCliTools(self.redpanda) | ||
| self.rpk = RpkTool(self.redpanda) | ||
|
|
||
| def tearDown(self): |
There was a problem hiding this comment.
No need to override if it's just calling the parent
|
|
||
| self._s3_port = si_settings.cloud_storage_api_endpoint_port | ||
|
|
||
| self.kafka_tools = KafkaCliTools(self.redpanda) |
|
Not I'm setting retention only after all segments are produced, also waiting for only one segment to be deleted. Now the test should be more stable. It passed 50 times out of 50 in CDT and 10 times out of 10 locally. Let's see how it behaves on CI |
When trying to consume from blocked S3, time out error should be returned
jcsp
approved these changes
Apr 29, 2022
|
Retried release-clang-arm64 for unrelated failure in coproc_fixture_rpunit |
|
Unrelated |
NyaliaLui
approved these changes
Apr 29, 2022
|
/backport v21.1.x |
|
/backport v22.1.x |
This was referenced May 9, 2022
mmedenjak
added
kind/enhance
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cover letter
When trying to consume from blocked S3, time out error should be
returned
Release notes