-
Notifications
You must be signed in to change notification settings - Fork 573
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enabling maintenance mode on a decommissioned node leaves the cluster in an inconsistent state #4999
Labels
Comments
nicolaferraro
added a commit
to nicolaferraro/redpanda
that referenced
this issue
Jun 10, 2022
nicolaferraro
added a commit
to nicolaferraro/redpanda
that referenced
this issue
Jun 10, 2022
nicolaferraro
added a commit
to nicolaferraro/redpanda
that referenced
this issue
Jun 10, 2022
nicolaferraro
added a commit
to nicolaferraro/redpanda
that referenced
this issue
Jun 13, 2022
nicolaferraro
added a commit
to nicolaferraro/redpanda
that referenced
this issue
Jun 14, 2022
…rkaround for redpanda-data#4999) When a node is shutdown after decommission, the maintenance mode hooks will trigger. While the process has no visible effect on partitions, it leaves the cluster in an inconsistent state, so that other nodes cannot enter maintenance mode. We force reset the flag with this change.
nicolaferraro
added a commit
to nicolaferraro/redpanda
that referenced
this issue
Jun 15, 2022
…rkaround for redpanda-data#4999) When a node is shutdown after decommission, the maintenance mode hooks will trigger. While the process has no visible effect on partitions, it leaves the cluster in an inconsistent state, so that other nodes cannot enter maintenance mode. We force reset the flag with this change.
nicolaferraro
added a commit
to nicolaferraro/redpanda
that referenced
this issue
Jun 16, 2022
…rkaround for redpanda-data#4999) When a node is shutdown after decommission, the maintenance mode hooks will trigger. While the process has no visible effect on partitions, it leaves the cluster in an inconsistent state, so that other nodes cannot enter maintenance mode. We force reset the flag with this change.
6 tasks
joejulian
pushed a commit
to joejulian/redpanda
that referenced
this issue
Mar 10, 2023
…node (workaround for redpanda-data#4999) When a node is shutdown after decommission, the maintenance mode hooks will trigger. While the process has no visible effect on partitions, it leaves the cluster in an inconsistent state, so that other nodes cannot enter maintenance mode. We force reset the flag with this change.
joejulian
pushed a commit
to joejulian/redpanda
that referenced
this issue
Mar 24, 2023
…g node (workaround for redpanda-data#4999) When a node is shutdown after decommission, the maintenance mode hooks will trigger. While the process has no visible effect on partitions, it leaves the cluster in an inconsistent state, so that other nodes cannot enter maintenance mode. We force reset the flag with this change.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What went wrong?
I'm trying maintenance mode hooks in combination with decommission and finding a strange behavior.
The problem is that maintenance mode can be enabled also for (fully) decommissioned nodes (static hooks do that automatically in k8s).
This causes two issues:
I think it's ok if the cluster replies (200) ok to a request to enter maintenance mode for a decommissioned node, but probably the internal state remains inconsistent.
cc @dotnwat
Version & Environment
Redpanda v22.1.3, latest operator
How to reproduce the issue?
This process has been executed on a 3 node
example
cluster:JIRA Link: CORE-933
The text was updated successfully, but these errors were encountered: