Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cloud_storage: tweak offsets used in post-recovery bootstrap #10535

Merged
merged 2 commits into from
Jun 5, 2023
Merged

cloud_storage: tweak offsets used in post-recovery bootstrap #10535

merged 2 commits into from
Jun 5, 2023

Conversation

VladLazar
Copy link
Contributor

@VladLazar VladLazar commented May 3, 2023 •
edited
Loading

For cloud based recovery, once segments are downloaded, the state for
raft, kvstore and local storage needs to be updated to reflect the
recovered log. Previously, the offsets were slightly off and this commit
adjusts them to always refer to the first/last offsets in the log.

Fixes #10219

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v23.1.x
  • v22.3.x
  • v22.2.x

Release Notes

Bug Fixes

  • Fix post-recovery Raft boostrap

@VladLazar VladLazar changed the title dummy commit [DO NOT MERGE] Runner for #10219 May 3, 2023
@VladLazar
Copy link
Contributor Author

/ci-repeat 5
skip-units
dt-repeat=10
tests/rptest/tests/topic_recovery_test.py::TopicRecoveryTest.test_empty_segments

1 similar comment
@VladLazar
Copy link
Contributor Author

/ci-repeat 5
skip-units
dt-repeat=10
tests/rptest/tests/topic_recovery_test.py::TopicRecoveryTest.test_empty_segments

@VladLazar
Copy link
Contributor Author

/ci-repeat 5
skip-units
dt-repeat=20
tests/rptest/tests/topic_recovery_test.py::TopicRecoveryTest.test_empty_segments

1 similar comment
@VladLazar
Copy link
Contributor Author

/ci-repeat 5
skip-units
dt-repeat=20
tests/rptest/tests/topic_recovery_test.py::TopicRecoveryTest.test_empty_segments

@VladLazar
Copy link
Contributor Author

/ci-repeat 5
skip-units
dt-repeat=20
tests/rptest/tests/topic_recovery_test.py::TopicRecoveryTest.test_empty_segments

@VladLazar
Copy link
Contributor Author

/ci-repeat 10
skip-units
dt-repeat=20
tests/rptest/tests/topic_recovery_test.py::TopicRecoveryTest.test_empty_segments

@VladLazar VladLazar changed the title [DO NOT MERGE] Runner for #10219 cloud_storage: tweak offsets used in post-recovery boostrap Jun 1, 2023
@VladLazar
Copy link
Contributor Author

/ci-repeat 10
skip-units
dt-repeat=5
tests/rptest/tests/topic_recovery_test.py::TopicRecoveryTest

@VladLazar VladLazar requested review from Lazin and andrwng June 1, 2023 12:27
@VladLazar VladLazar marked this pull request as ready for review June 1, 2023 12:27
@VladLazar VladLazar changed the title cloud_storage: tweak offsets used in post-recovery boostrap cloud_storage: tweak offsets used in post-recovery bootstrap Jun 1, 2023
c/consensus_utils: adjust recovery offset ranges
ceeeb3d 
For cloud based recovery, once segments are downloaded, the state for
raft, kvstore and local storage needs to be updated to reflect the
recovered log. Previously, the offsets were slightly off and this commit
adjusts them to always refer to the first/last offsets in the log.
@VladLazar
Copy link
Contributor Author

/ci-repeat 10
skip-units
dt-repeat=10
tests/rptest/tests/topic_recovery_test.py::TopicRecoveryTest

@VladLazar
Copy link
Contributor Author

/ci-repeat 10
skip-units
dt-repeat=20
tests/rptest/tests/topic_recovery_test.py::TopicRecoveryTest

@VladLazar
Copy link
Contributor Author

All tests passed. One of the debug build jobs failed to build the ducktape image:

minio Error 
Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: TLS handshake timeout
task: Failed to run task "rp:run-ducktape-tests": task: Failed to run task "rp:init-ducktape-containers": task: Failed to run task "rp:start-compose-cluster": exit status 18

@piyushredpanda piyushredpanda merged commit 3c2b7dd into redpanda-data:dev Jun 5, 2023
@vbotbuildovich
Copy link
Collaborator

/backport v23.1.x

@vbotbuildovich
Copy link
Collaborator

Failed to run cherry-pick command. I executed the commands below:

git checkout -b backport-pr-10535-v23.1.x-36 remotes/upstream/v23.1.x
git cherry-pick -x 206ceef3f50ff89312fdcfb12ebd2a760481695a ceeeb3d8af5b4a4fbd75e1be1d058617583998fa

Workflow run logs.

@vbotbuildovich vbotbuildovich mentioned this pull request Jun 5, 2023
Closed
@VladLazar VladLazar mentioned this pull request Jun 5, 2023
Merged
7 tasks
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ztlpn ztlpn ztlpn approved these changes

@Lazin Lazin Awaiting requested review from Lazin

@andrwng andrwng Awaiting requested review from andrwng

Assignees
No one assigned
Labels
area/redpanda
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

CI Failure (timeout) in TopicRecoveryTest.test_empty_segments
4 participants
@VladLazar @vbotbuildovich @ztlpn @piyushredpanda