Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Remote Segments] Avoid segments metadata upload if no difference in local store and remote store #7443

Closed
ashking94 opened this issue May 5, 2023 · 0 comments · Fixed by #7905
Closed

[Remote Segments] Avoid segments metadata upload if no difference in local store and remote store #7443

ashking94 opened this issue May 5, 2023 · 0 comments · Fixed by #7905
Assignees
@sachinpkale
Labels
enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework v2.8.0 'Issues and PRs related to version v2.8.0'

Comments

@ashking94
Copy link
Member

Is your feature request related to a problem? Please describe.
Need to look at 2 issues -

  1. Currently in afterRefresh method of RemoteStoreRefreshListener, we upload metadata file even if the local and remote store has not difference in segment files. We need to skip the upload if there are no differences in segment files on local and remote store.
  2. Noticing that the afterRefresh method gets invoked twice - once with didRefresh true and other time with false. This leads to minSeqNoToKeep getting set correctly in the second run. Need to fix this as well.
@ashking94 ashking94 added enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework v2.8.0 'Issues and PRs related to version v2.8.0' labels May 5, 2023
@sachinpkale sachinpkale mentioned this issue May 5, 2023
Closed
13 tasks
@gbbafna gbbafna removed the untriaged label May 16, 2023
@sachinpkale sachinpkale self-assigned this May 22, 2023
This was referenced May 26, 2023
Closed
Merged
@ashking94 ashking94 mentioned this issue Jun 3, 2023
Merged
6 tasks
@ashking94 ashking94 mentioned this issue Jul 3, 2023
Merged
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sachinpkale sachinpkale

Labels
enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework v2.8.0 'Issues and PRs related to version v2.8.0'
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Remote Store] Optimize segments metadata upload ashking94/OpenSearch
3 participants
@sachinpkale @ashking94 @gbbafna