Add request snapshot #243

overvenus · 2019-06-10T07:12:28Z

This adds a feature that allows follower/learner request snapshot actively.

Impls tikv/rfcs#26

Signed-off-by: Neil Shen <overvenus@gmail.com>

… already Signed-off-by: Neil Shen <overvenus@gmail.com>

Signed-off-by: Neil Shen <overvenus@gmail.com>

proto/eraftpb.proto

src/errors.rs

src/progress.rs

src/raft.rs

Signed-off-by: Neil Shen <overvenus@gmail.com>

src/progress.rs

src/raft.rs

Signed-off-by: Neil Shen <overvenus@gmail.com>

src/raft.rs

src/storage.rs

tests/integration_cases/test_raft.rs

Signed-off-by: Neil Shen <overvenus@gmail.com>

src/raft.rs

hicqu · 2019-06-27T11:45:52Z

src/progress.rs

+            if self.next_idx < 1 {
+                self.next_idx = 1;
+            }
+        } else if self.pending_request_snapshot == INVALID_INDEX {


Why else if instead of else?

We don't want to update pending request snapshot if there is one already, let's keep it simple.

hicqu · 2019-06-27T11:49:11Z

src/raft.rs

    // TODO: revoke pub when there is a better way to test.
    /// For a given message, append the entries to the log.
    pub fn handle_append_entries(&mut self, m: &Message) {
+        if self.pending_request_snapshot != INVALID_INDEX {


what if the entries is more new than pending_request_snapshot?

It should be dropped if a follower is requesting a snapshot. After applying a snapshot, the follower will receive raft log entries again.

hicqu · 2019-06-27T11:50:41Z

src/raft.rs

@@ -1794,7 +1849,10 @@ impl<T: Storage> Raft<T> {

    fn restore_raft(&mut self, snap: &Snapshot) -> Option<bool> {
        let meta = snap.get_metadata();
-        if self.raft_log.match_term(meta.get_index(), meta.get_term()) {
+        // Do not fast-forward commit if we are requesting snapshot.


Because the snapshot may be dropped if there is no further proposal after requesting a snapshot.

zhangjinpeng87 · 2019-06-29T00:47:17Z

src/progress.rs

        if self.state == ProgressState::Replicate {
            // the rejection must be stale if the progress has matched and "rejected"
            // is smaller than "match".
-            if rejected <= self.matched {
+            // Or rejected equals to matched and request_snapshot is the INVALID_INDEX.


Can you give more explanation for this condition?

src/raft.rs

zhangjinpeng87 · 2019-06-29T01:15:09Z

src/raft.rs

+        let mut m = Message::new();
+        m.set_msg_type(MessageType::MsgAppendResponse);
+        m.set_index(self.raft_log.committed);
+        m.set_reject(true);


Why set reject?

It needs to be rejected, so it can enter requesting snapshot state in leader's view. See more at handle_append_response.

zhangjinpeng87 · 2019-06-29T01:16:50Z

src/raft.rs

    // TODO: revoke pub when there is a better way to test.
    /// For a given message, append the entries to the log.
    pub fn handle_append_entries(&mut self, m: &Message) {
+        if self.pending_request_snapshot != INVALID_INDEX {
+            self.send_request_snapshot();


Does the leader can tolerant follower send_request_snapshot multiple times with the same pending_request_snapshot?

Yes, because request snapshot message may be dropped on the network, it needs to be able to send the same message multiple times.

src/storage.rs

Signed-off-by: Neil Shen <overvenus@gmail.com>

zhangjinpeng87

LGTM

overvenus · 2019-07-01T06:36:33Z

PTAL @hicqu @BusyJay Thanks!

Signed-off-by: Nick Cameron <nrc@ncameron.org>

Signed-off-by: Neil Shen <overvenus@gmail.com>

* Check pending conf change before campaign (#225) Fix #221. * Add more convenient lite-weight interfaces (#227) This PR introduces two simple and lite weight interfaces: - ping to trigger heartbeats without ticking, - status_ref to borrow the progress set instead of cloning. * *: bump to 0.4.2 (#228) * Bump to v0.4.3 (#231) * raft: leader respond to learner read index message (#220) Signed-off-by: nolouch <nolouch@gmail.com> * Bump to v0.4.3 Signed-off-by: Neil Shen <overvenus@gmail.com> * Request snapshot (#243) Signed-off-by: Neil Shen <overvenus@gmail.com> * fix tests * cargo fmt * address comments.

overvenus added 8 commits May 26, 2019 23:14

request snapshot

3c5e02c

Signed-off-by: Neil Shen <overvenus@gmail.com>

Export request snapshot method

b59e2e4

Signed-off-by: Neil Shen <overvenus@gmail.com>

Fix CI

0a30175

Signed-off-by: Neil Shen <overvenus@gmail.com>

request_snapshot also affects flow control

f1d5104

Signed-off-by: Neil Shen <overvenus@gmail.com>

Can not request snapshot if there is no leader or there is a snapshot…

e7a534b

… already Signed-off-by: Neil Shen <overvenus@gmail.com>

Follower can not become leader during requesting snaphot

0222aa7

Signed-off-by: Neil Shen <overvenus@gmail.com>

send append if peer is requesting snapshot

4c8a95d

cargo fmt

fbf8981

Signed-off-by: Neil Shen <overvenus@gmail.com>

overvenus requested review from BusyJay and hicqu June 10, 2019 07:12

overvenus added 5 commits June 10, 2019 19:57

remove requesting snapshot flag

7048c19

Signed-off-by: Neil Shen <overvenus@gmail.com>

Add request snapshot field in message

89f32e1

Signed-off-by: Neil Shen <overvenus@gmail.com>

Allow starting leader election in requesting snapshot state

a2e038d

Signed-off-by: Neil Shen <overvenus@gmail.com>

Remove MsgRequestSnapshot message type

e087dd9

Signed-off-by: Neil Shen <overvenus@gmail.com>

Try to fix CI

70949a0

Signed-off-by: Neil Shen <overvenus@gmail.com>

BusyJay reviewed Jun 13, 2019

View reviewed changes

proto/eraftpb.proto Outdated Show resolved Hide resolved

src/errors.rs Outdated Show resolved Hide resolved

src/progress.rs Outdated Show resolved Hide resolved

src/raft.rs Outdated Show resolved Hide resolved

src/raft.rs Outdated Show resolved Hide resolved

overvenus added 6 commits June 14, 2019 15:04

Pass a request index to the snapshot method

197494d

Signed-off-by: Neil Shen <overvenus@gmail.com>

Address comments

3897ed5

Signed-off-by: Neil Shen <overvenus@gmail.com>

update request_snpashot doc

0648140

Signed-off-by: Neil Shen <overvenus@gmail.com>

Request snapshot on heartbeat

e9880d4

Signed-off-by: Neil Shen <overvenus@gmail.com>

Alway consider decreased if the peer is requesting snapshot

43b6c93

Signed-off-by: Neil Shen <overvenus@gmail.com>

Add two tests

2cdc800

Signed-off-by: Neil Shen <overvenus@gmail.com>

overvenus mentioned this pull request Jun 19, 2019

raftstore: support request snapshot on leader's side tikv/tikv#4926

Merged

BusyJay reviewed Jun 19, 2019

View reviewed changes

src/progress.rs Show resolved Hide resolved

BusyJay reviewed Jun 19, 2019

View reviewed changes

src/raft.rs Outdated Show resolved Hide resolved

BusyJay reviewed Jun 19, 2019

View reviewed changes

src/raft.rs Outdated Show resolved Hide resolved

BusyJay reviewed Jun 19, 2019

View reviewed changes

src/raft.rs Show resolved Hide resolved

BusyJay reviewed Jun 19, 2019

View reviewed changes

src/raft.rs Outdated Show resolved Hide resolved

overvenus added 2 commits June 19, 2019 21:03

Address comments

fd4edad

Signed-off-by: Neil Shen <overvenus@gmail.com>

Allow request snapshot when it's state is not Replicate

ef84d14

Signed-off-by: Neil Shen <overvenus@gmail.com>

BusyJay reviewed Jun 21, 2019

View reviewed changes

src/raft.rs Outdated Show resolved Hide resolved

src/raft.rs Show resolved Hide resolved

src/storage.rs Show resolved Hide resolved

tests/integration_cases/test_raft.rs Show resolved Hide resolved

overvenus added 4 commits June 24, 2019 18:50

Do not consider a request snapshot message stale

4336598

Signed-off-by: Neil Shen <overvenus@gmail.com>

Address comments

c8d4854

Signed-off-by: Neil Shen <overvenus@gmail.com>

Address comments

6191ccb

Signed-off-by: Neil Shen <overvenus@gmail.com>

Check index for snapshot in raft log unstable

1448df7

Signed-off-by: Neil Shen <overvenus@gmail.com>

BusyJay previously approved these changes Jun 27, 2019

View reviewed changes

hicqu reviewed Jun 27, 2019

View reviewed changes

src/raft.rs Show resolved Hide resolved

hicqu reviewed Jun 27, 2019

View reviewed changes

zhangjinpeng87 reviewed Jun 29, 2019

View reviewed changes

src/raft.rs Outdated Show resolved Hide resolved

zhangjinpeng87 reviewed Jun 29, 2019

View reviewed changes

src/storage.rs Outdated Show resolved Hide resolved

Address comments

a2a5c53

Signed-off-by: Neil Shen <overvenus@gmail.com>

overvenus dismissed BusyJay’s stale review via a2a5c53 June 29, 2019 08:17

zhangjinpeng87 approved these changes Jun 29, 2019

View reviewed changes

BusyJay approved these changes Jul 1, 2019

View reviewed changes

overvenus merged commit fe10576 into 0.4.x Jul 1, 2019

nrc added a commit to nrc/raft-rs that referenced this pull request Jul 11, 2019

Port tikv#243 to the Prost branch

da05916

Signed-off-by: Nick Cameron <nrc@ncameron.org>

nrc added a commit to nrc/raft-rs that referenced this pull request Jul 11, 2019

Port tikv#243 to the Prost branch

fa03acc

Signed-off-by: Nick Cameron <nrc@ncameron.org>

nrc mentioned this pull request Jul 11, 2019

Port #243 to the Prost branch #259

Merged

nrc added a commit to nrc/raft-rs that referenced this pull request Jul 11, 2019

Port tikv#243 to the Prost branch

66c9c1e

Signed-off-by: Nick Cameron <nrc@ncameron.org>

nrc added a commit that referenced this pull request Jul 12, 2019

Port #243 to the Prost branch (#259)

eeaee89

Signed-off-by: Nick Cameron <nrc@ncameron.org>

hicqu pushed a commit to hicqu/raft-rs that referenced this pull request Jul 17, 2019

Request snapshot (tikv#243)

e6d98a4

Signed-off-by: Neil Shen <overvenus@gmail.com>

BusyJay mentioned this pull request Mar 3, 2020

*: bump 0.4.4 #341

Merged

overvenus deleted the request-snapshot branch October 15, 2023 10:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add request snapshot #243

Add request snapshot #243

overvenus commented Jun 10, 2019

hicqu Jun 27, 2019

overvenus Jun 27, 2019

hicqu Jun 27, 2019

overvenus Jun 27, 2019

hicqu Jun 27, 2019

overvenus Jun 27, 2019

zhangjinpeng87 Jun 29, 2019

zhangjinpeng87 Jun 29, 2019

zhangjinpeng87 Jun 29, 2019

overvenus Jun 29, 2019

zhangjinpeng87 Jun 29, 2019

overvenus Jun 29, 2019

zhangjinpeng87 left a comment

overvenus commented Jul 1, 2019

Add request snapshot #243

Add request snapshot #243

Conversation

overvenus commented Jun 10, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhangjinpeng87 left a comment

Choose a reason for hiding this comment

overvenus commented Jul 1, 2019