Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store: Make bucket store initial sync more robust #6050

Merged
merged 1 commit into from
Jan 27, 2023

Conversation

Kartik-Garg
Copy link
Contributor

@Kartik-Garg Kartik-Garg commented Jan 17, 2023

This PR closes #4810

Signed-off-by: Kartik-Garg kartik.garg@infracloud.io

  • I added CHANGELOG entry for this change.
  • Change is not relevant to the end user.

Changes

This PR implements the change: If the initial sync fails for the store, it will retry the initialization using runutil.retry with default values, it retries to initialize the store every 10 seconds for 30 duration of seconds, i.e., total of three times

Verification

After starting the local minio server, started store component (quickstart gist, edited from quickstart.sh)
Store retried sync and succeeded with initial sync

@Kartik-Garg Kartik-Garg force-pushed the store-initial-sync-robust branch 7 times, most recently from cc05e2d to e85223b Compare January 18, 2023 07:20
@Kartik-Garg Kartik-Garg force-pushed the store-initial-sync-robust branch 9 times, most recently from 455524f to e20ef21 Compare January 19, 2023 07:16
@Kartik-Garg Kartik-Garg changed the title WIP: Store - Make initial sync more robust Store - Make initial sync more robust Jan 19, 2023
@Kartik-Garg Kartik-Garg changed the title Store - Make initial sync more robust Store: Make bucket store initial sync more robust Jan 19, 2023
@Kartik-Garg Kartik-Garg force-pushed the store-initial-sync-robust branch 6 times, most recently from 9736806 to 94d249b Compare January 20, 2023 16:32
@Kartik-Garg
Copy link
Contributor Author

@matej-g could someone from team please review this PR? :)

@pull-request-size pull-request-size bot added size/M and removed size/S labels Jan 24, 2023
@pull-request-size pull-request-size bot added size/S and removed size/M labels Jan 24, 2023
@Kartik-Garg Kartik-Garg force-pushed the store-initial-sync-robust branch 4 times, most recently from 4a56ae0 to 4eed912 Compare January 24, 2023 12:32
Copy link
Contributor

@fpetkovski fpetkovski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some small nits from my side.

cmd/thanos/store.go Outdated Show resolved Hide resolved
cmd/thanos/store.go Outdated Show resolved Hide resolved
cmd/thanos/store.go Outdated Show resolved Hide resolved
@Kartik-Garg
Copy link
Contributor Author

Just some small nits from my side.

@fpetkovski , thanks for the suggestions, I have made the changes accordingly :)

@Kartik-Garg Kartik-Garg force-pushed the store-initial-sync-robust branch 3 times, most recently from dc58870 to a5972ea Compare January 25, 2023 10:59
cmd/thanos/store.go Outdated Show resolved Hide resolved
Added re-try mechanism for store inital sync, where if the initial sync fails, it tries to do the initial sync again for given timeout duration.

Signed-off-by: Kartik-Garg <kartik.garg@infracloud.io>
@Kartik-Garg Kartik-Garg requested review from matej-g and fpetkovski and removed request for matej-g and fpetkovski January 26, 2023 12:54
Copy link
Collaborator

@matej-g matej-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @Kartik-Garg! 😊

@Kartik-Garg
Copy link
Contributor Author

Thank you @Kartik-Garg! 😊

@matej-g thanks a lot for approving it, this is going to be my first open source PR (non document related) 🎉.
If possible can you also please merge this PR?

@fpetkovski fpetkovski merged commit 24226b1 into thanos-io:main Jan 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Store: Make initial sync more robust
3 participants