Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch split for prewrite and commit requests #390

Merged
merged 1 commit into from
Feb 8, 2023

Conversation

akoshchiy
Copy link
Contributor

ref #369
Still in progress, here is draft api for splitting/merging requests into batches (RawBatchPutRequest as example).
After approve i will finish for the rest.

@akoshchiy akoshchiy marked this pull request as draft January 29, 2023 10:15
@akoshchiy
Copy link
Contributor Author

PTAL @ekexium

@ekexium
Copy link
Collaborator

ekexium commented Jan 29, 2023

Hey thank you for the work!
I think we'd better not split a raw request which could make it non-atomic.
I guess supporting prewrite and commit requests in transactional API would be enough for us.
And we need a test to verify that it works

@akoshchiy
Copy link
Contributor Author

akoshchiy commented Jan 29, 2023

@ekexium Since we don't need reusable api for batches, is it ok to implement batching for prewrite/commit requests directly in Shardable trait implementation?
As an example, if we have two regions and two batches inside each region, shards() method will return four "shards".
So, it won't be necessary to merge batch responses, and other logic in request pipeline will be untouched.

@ekexium
Copy link
Collaborator

ekexium commented Jan 30, 2023

@ekexium Since we don't need reusable api for batches, is it ok to implement batching for prewrite/commit requests directly in Shardable trait implementation? As an example, if we have two regions and two batches inside each region, shards() method will return four "shards". So, it won't be necessary to merge batch responses, and other logic in request pipeline will be untouched.

Yes we can

Signed-off-by: Andrey Koshchiy <roguepnz@gmail.com>
@akoshchiy akoshchiy force-pushed the feature/369-tx-batch-mutations branch from c8748ed to 716540f Compare February 1, 2023 20:43
@akoshchiy akoshchiy changed the title [WIP] split requests into batches Batch split for prewrite and commit requests Feb 1, 2023
@akoshchiy akoshchiy marked this pull request as ready for review February 1, 2023 20:52
@akoshchiy
Copy link
Contributor Author

@ekexium PTAL.
Added raft-entry-max-size to the config to be able to test with small amount of data.

@ekexium ekexium merged commit 8f54e61 into tikv:master Feb 8, 2023
bxq2011hust pushed a commit to FISCO-BCOS/tikv-client-rust that referenced this pull request Mar 20, 2024
Signed-off-by: Andrey Koshchiy <roguepnz@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants