[backport] [v23.1.x] More detailed partition reconfiguration tracking #10201 #10630
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bharathv
changed the title
ztlpn
reviewed
May 9, 2023
| @@ -273,6 +273,29 @@ inline bool has_non_replicable_op_type(const topic_table_delta& d) { | |||
| } | |||
| __builtin_unreachable(); | |||
| } | |||
|
|
|||
| inline std::vector<model::broker_shard> union_replica_sets( | |||
There was a problem hiding this comment.
this is actually needed by the previous commit
There was a problem hiding this comment.
I thought no one would notice it in a backport :-P
Signed-off-by: Michal Maslanka <michal@redpanda.com> (cherry picked from commit cd45527)
In order to provide a generic error code to express errors originating from outside of the cluster module (errors with different category) or an exceptions occurred in `controller_backend` we introduce a separate error code. Signed-off-by: Michal Maslanka <michal@redpanda.com> (cherry picked from commit 711949d)
Signed-off-by: Michal Maslanka <michal@redpanda.com> (cherry picked from commit 8f83991)
Added revision, last error and retry count to backend operation. The information will be used to track partition reconfiguration progress. Signed-off-by: Michal Maslanka <michal@redpanda.com> (cherry picked from commit 8e01dd0)
Added `controller_api` that allows caller to request partition reconciliation state from all the replicas where partition is currently hosted. The API returns a data structure containing operations that are executed by `controller_backend` on all of the replicas. Signed-off-by: Michal Maslanka <michal@redpanda.com> (cherry picked from commit 1799693)
The `/reconfiguartions` endpoint didn't provide an insight into the progress of partition reconfigurations. Added information that will allow user to check the operation progress and additionally check status of reconciliation on all replicas. Signed-off-by: Michal Maslanka <michal@redpanda.com> (cherry picked from commit b2467b3)
Signed-off-by: Michal Maslanka <michal@redpanda.com> (cherry picked from commit d83a975)
ztlpn
approved these changes
May 10, 2023
|
Failure: #10219 (known issue, unrelated). |
This pull request was closed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Enriched
/reconfigurationAPI with more information allowing users to track the progress of partition reconciliation. Now the API returns a complete set of information related with partition reconfiguration that is taking place.The API will now return the following JSON:
{ "ns": "kafka", "topic": "topic-khbikkrzeo", "partition": 9, "previous_replicas": [ { "node_id": 2, "core": 0 }, { "node_id": 3, "core": 0 }, { "node_id": 1, "core": 0 } ], "current_replicas": [ { "node_id": 4, "core": 0 }, { "node_id": 3, "core": 0 }, { "node_id": 1, "core": 0 } ], "bytes_left_to_move": 190, "bytes_moved": 0, "partition_size": 190, "reconciliation_statuses": [ { "node_id": 2, "operations": [ { "type": "update", "core": 0, "retry_number": 7, "revision": 89, "status": "Generic failure occurred during partition operation execution (cluster::errc:52)" } ] }, { "node_id": 1, "operations": [ { "type": "update", "core": 0, "retry_number": 3, "revision": 89, "status": "Current node is not a leader for partition (cluster::errc:17)" } ] }, { "node_id": 4, "operations": [ { "type": "update", "core": 0, "retry_number": 5, "revision": 89, "status": "Current node is not a leader for partition (cluster::errc:17)" } ] }, { "node_id": 3, "operations": [ { "type": "update", "core": 0, "retry_number": 5, "revision": 89, "status": "Current node is not a leader for partition (cluster::errc:17)" } ] } ] }Fixes #10434
Backport of #10201
Backports Required
Release Notes