Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Skip zone/host awareness when auto-expand set to "all" #14603

Open
amberzsy opened this issue Jun 30, 2024 · 1 comment · May be fixed by #14619
Open

[Feature Request] Skip zone/host awareness when auto-expand set to "all" #14603

amberzsy opened this issue Jun 30, 2024 · 1 comment · May be fixed by #14619
Labels
enhancement Enhancement or improvement to existing feature or request ShardManagement:Resiliency

Comments

@amberzsy
Copy link

Is your feature request related to a problem? Please describe

Today, if index is set to index.auto_expand_replicas with upper bound to "all", the cluster will automatically expand to same # of replicas as # of data nodes. This is will easily conflicts with shard allocation awareness if there's an uneven distribution of nodes across zones or hosts, which cause some unassigned shards(cluster yellow state).

Describe the solution you'd like

It will be good to ignore the same hots allocation decider and allocation awareness decider if using auto expand to all feature. As both actually aim to achieve resilience goal of avoiding uneven or concentration of shards on single failure host or zone, with all copies distributed to every nodes, it's already achieve the goal and zone/rack awareness cannot improve the resilience but only create conflict.

Related component

ShardManagement:Resiliency

Describe alternatives you've considered

No response

Additional context

No response

@amberzsy amberzsy added enhancement Enhancement or improvement to existing feature or request untriaged labels Jun 30, 2024
@amberzsy amberzsy linked a pull request Jul 2, 2024 that will close this issue
3 tasks
@peternied
Copy link
Member

[Triage - attendees 1 2 3]
@amberzsy Thanks for creating this issue and a pull request along side it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request ShardManagement:Resiliency
Projects
Status: 🆕 New
Development

Successfully merging a pull request may close this issue.

2 participants