Failure in maintenance_test.MaintenanceTest.test_maintenance_sticky

[gousteris]

bk ref: https://buildkite.com/redpanda/redpanda/builds/8738#77988e09-d58a-4f15-8132-0dd6e74090b1/1548-8060

test_id:    rptest.tests.maintenance_test.MaintenanceTest.test_maintenance_sticky
--
  | status:     FAIL
  | run time:   1 minute 47.688 seconds
  |  
  |  
  | TimeoutError('')
  | Traceback (most recent call last):
  | File "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 135, in run
  | data = self.run_test()
  | File "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 227, in run_test
  | return self.test_context.function(self.test)
  | File "/root/tests/rptest/services/cluster.py", line 35, in wrapped
  | r = f(self, *args, **kwargs)
  | File "/root/tests/rptest/tests/maintenance_test.py", line 120, in test_maintenance_sticky
  | self._enable_maintenance(node)
  | File "/root/tests/rptest/tests/maintenance_test.py", line 81, in _enable_maintenance
  | wait_until(lambda: not self._has_leadership_role(node),
  | File "/usr/local/lib/python3.9/dist-packages/ducktape/utils/util.py", line 58, in wait_until
  | raise TimeoutError(err_msg() if callable(err_msg) else err_msg) from last_exception
  | ducktape.errors.TimeoutError
  |  

[jcsp]

Is this the same basic thing as #4236 ?

[mmedenjak]

Failed again in https://buildkite.com/redpanda/redpanda/builds/12848#01822070-a013-499c-be55-e8f7c4431e02:

[INFO:2022-07-21 13:01:52,944]: RunnerClient: rptest.tests.maintenance_test.MaintenanceTest.test_maintenance_sticky.use_rpk=False: FAIL: AssertionError('`rpk cluster maintenance status` has changed: []')
--
  | Traceback (most recent call last):
  | File "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 135, in run
  | data = self.run_test()
  | File "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 227, in run_test
  | return self.test_context.function(self.test)
  | File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 476, in wrapper
  | return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
  | File "/root/tests/rptest/services/cluster.py", line 35, in wrapped
  | r = f(self, *args, **kwargs)
  | File "/root/tests/rptest/tests/maintenance_test.py", line 234, in test_maintenance_sticky
  | self._verify_cluster(node, True)
  | File "/root/tests/rptest/tests/maintenance_test.py", line 190, in _verify_cluster
  | wait_until(
  | File "/usr/local/lib/python3.9/dist-packages/ducktape/utils/util.py", line 53, in wait_until
  | raise e
  | File "/usr/local/lib/python3.9/dist-packages/ducktape/utils/util.py", line 44, in wait_until
  | if condition():
  | File "/root/tests/rptest/tests/maintenance_test.py", line 191, in <lambda>
  | lambda: self._verify_maintenance_status(node, expect),
  | File "/root/tests/rptest/tests/maintenance_test.py", line 102, in _verify_maintenance_status
  | statuses = self.rpk.cluster_maintenance_status()
  | File "/root/tests/rptest/clients/rpk.py", line 624, in cluster_maintenance_status
  | return list(filter(None, map(parse, output.splitlines())))
  | File "/root/tests/rptest/clients/rpk.py", line 599, in parse
  | assert len(
  | AssertionError: `rpk cluster maintenance status` has changed: []
  |  
  | [INFO:2022-07-21 13:01:52,945]: RunnerClient: rptest.tests.maintenance_test.MaintenanceTest.test_maintenance_sticky.use_rpk=False: Tearing down...
  | [INFO:2022-07-21 13:02:00,607]: RunnerClient: rptest.tests.prefix_truncate_recovery_test.PrefixTruncateRecoveryTest.test_prefix_truncate_recovery.acks=1.start_empty=True: Running...
  | [INFO:2022-07-21 13:02:01,021]: RunnerClient: rptest.tests.maintenance_test.MaintenanceTest.test_maintenance_sticky.use_rpk=False: Summary: AssertionError('`rpk cluster maintenance status` has changed: []')
  | Traceback (most recent call last):
  | File "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 135, in run
  | data = self.run_test()
  | File "/usr/local/lib/python3.9/dist-packages/ducktape/tests/runner_client.py", line 227, in run_test
  | return self.test_context.function(self.test)
  | File "/usr/local/lib/python3.9/dist-packages/ducktape/mark/_mark.py", line 476, in wrapper
  | return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
  | File "/root/tests/rptest/services/cluster.py", line 35, in wrapped
  | r = f(self, *args, **kwargs)
  | File "/root/tests/rptest/tests/maintenance_test.py", line 234, in test_maintenance_sticky
  | self._verify_cluster(node, True)
  | File "/root/tests/rptest/tests/maintenance_test.py", line 190, in _verify_cluster
  | wait_until(
  | File "/usr/local/lib/python3.9/dist-packages/ducktape/utils/util.py", line 53, in wait_until
  | raise e
  | File "/usr/local/lib/python3.9/dist-packages/ducktape/utils/util.py", line 44, in wait_until
  | if condition():
  | File "/root/tests/rptest/tests/maintenance_test.py", line 191, in <lambda>
  | lambda: self._verify_maintenance_status(node, expect),
  | File "/root/tests/rptest/tests/maintenance_test.py", line 102, in _verify_maintenance_status
  | statuses = self.rpk.cluster_maintenance_status()
  | File "/root/tests/rptest/clients/rpk.py", line 624, in cluster_maintenance_status
  | return list(filter(None, map(parse, output.splitlines())))
  | File "/root/tests/rptest/clients/rpk.py", line 599, in parse
  | assert len(
  | AssertionError: `rpk cluster maintenance status` has changed: []

[jcsp]

@mmedenjak that failure is #4772 (which is now fixed, the linked run was on an older commit)