Raft flush fixes #2937
Raft flush fixes #2937
Conversation
When truncating log we need to update raft flushed offset to reflect truncated log state Signed-off-by: Michal Maslanka <michal@vectorized.io>
mmaslankaprv
force-pushed
the
raft-flush-fixes
branch
2 times, most recently
from
5c37af5
to
b1f2142
Compare
mmaslankaprv
requested review from
jcsp,
rystsov,
VadimPlh and
ztlpn
as code owners
|
These changes look good to me, although I think we're still going to have a very far-behind _flush_offset on followers when doing acks=1 writes, right? That is harmless as long as _commit_offset on the leader is only essential to these types of situation, but if we are e.g. exposing _commit_offset as a statistic, it's going to look pretty weird to human eyes. I think it would make sense to also auto-flush followers when they are more than N bytes behind the head of the log, even if it's not strictly necessary for the raft code. But that could be a separate PR, as I'd like to merge this one promptly to resolve the failing test. |
I agree that we should periodically flush on followers. The committed_index is never exposed to Kafka API so this should not be misleading to users. I know @rystsov is going to work on periodic flushes in raft |
mmaslankaprv
force-pushed
the
raft-flush-fixes
branch
from
b1f2142
to
b4a6b1b
Compare
Signed-off-by: Michal Maslanka <michal@vectorized.io>
In redpanda raft implementation supports relaxed consistency. With relaxed consistency we not always update `_commit_index` even tho entries were replicated to all the nodes. When performing leadership transfer we need to take that into account. When configuration is successfully replicated to majority of nodes and it is not in joint consensus state we are safe to proceed with the transfer. Signed-off-by: Michal Maslanka <michal@vectorized.io>
mmaslankaprv
force-pushed
the
raft-flush-fixes
branch
from
b4a6b1b
to
42d3d54
Compare
Cover letter
Fixed tracking flushed offset in an events of log truncation (both prefix and suffix). Fixed checking if configuration update is taking place when transferring leadership.
Fixes: #2932
Release notes