status api: Get engine type api #11
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
close tikv#13941, ref pingcap/tidb#39620 - If failed to get initial snapshot, remove the subscription as soon as possible. - Added a cache of getting checkpoint. This cache is lease-based -- the lease time is simply the tick interval of the coordinator. - Make the channel size huger for don't blocking the main loop when many regions migrating. Signed-off-by: hillium <yujuncen@pingcap.com> Signed-off-by: hillium <yu745514916@live.com> Signed-off-by: 山岚 <36239017+YuJuncen@users.noreply.github.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
…v#13984) ref tikv#12842 Signed-off-by: tabokie <xy.tao@outlook.com>
ref tikv#12842 Signed-off-by: tabokie <xy.tao@outlook.com> Signed-off-by: Xinye Tao <xy.tao@outlook.com>
ref tikv#12842 Implement periodical purge in v2. Signed-off-by: tabokie <xy.tao@outlook.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12842 If operations like snapshot, split, are aborted by restart, they needs to be either resumed or cleanup. This PR checks for garbage after restart and resume committed operations. Signed-off-by: Jay Lee <BusyJayLee@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> Signed-off-by: Zwb <ethercflow@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12842 a few panic fix 1) update_approximate_raft_log_size may run into divid by zero error 2) appy_delete may have None write_batch 3) StoreMeta::set_region may run into region corruption error if it's destroyed and re-created. 4) TabletSnapManager's snapshot size calculation may throw Other error. Signed-off-by: qi.xu <tonxuqi@outlook.com> Signed-off-by: Jay Lee <BusyJayLee@gmail.com> Co-authored-by: qi.xu <tonxuqi@outlook.com> Co-authored-by: Jay Lee <BusyJayLee@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
…13995) ref tikv#12842 Whenever timeout, the peer will check for unapplied logs whether there are pending conf change and trigger heavy reads. So we wait till most logs are applied before ticking. It also fix following issues: - PersistenceListener is not installed - implementation of persisted_apply_index is wrong - parse tablet name is wrong Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
ref tikv#12842 Signed-off-by: bufferflies <1045931706@qq.com>
close tikv#13997 Support to use evict_entry_cache when restart node. Signed-off-by: tabokie <xy.tao@outlook.com> Signed-off-by: hongyunyan <649330952@qq.com> Signed-off-by: Xinye Tao <xy.tao@outlook.com> Signed-off-by: Jay Lee <BusyJayLee@gmail.com> Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> Signed-off-by: Zwb <ethercflow@gmail.com> Co-authored-by: Xinye Tao <xy.tao@outlook.com> Co-authored-by: Jay <BusyJay@users.noreply.github.com> Co-authored-by: Zwb <ethercflow@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12842 The API is supposed to be used with `append` but nowhere can we find the clue. This PR merges `cut_logs` and `append` to reduce confusion and mistakes. Signed-off-by: Jay Lee <BusyJayLee@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12842 Publish tablet in apply thread is unsafe. This PR moves the operation to raftstore. It also fixes the issues that applying two splits at a time can cause panic. It also makes sure cache will be cleared after tablet is published. Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
ref tikv#12842 These two are helpers to utilize the static KV pairs in logger. In the past, we use `logger.list()` to try to format the configured KV pairs, but it will not work as values are omitted. Signed-off-by: Jay Lee <BusyJayLee@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#13730 Introduce priority-based channel Signed-off-by: Connor1996 <zbk602423539@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12842 When the tablet contains dirty data right after split, generating snapshot may just a waste. On the other hand, split usually happens on all peers, so delay it a bit actually makes all peers more likely to be initialized by split. So this PR rejects generating snapshot when it detects it still has dirty data. Signed-off-by: Jay Lee <BusyJayLee@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12842 Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
ref tikv#12842 1. store heartbeat should add snapshot and kv engine used size Signed-off-by: bufferflies <1045931706@qq.com> Co-authored-by: Xinye Tao <xy.tao@outlook.com>
close tikv#13977 Signed-off-by: tabokie <xy.tao@outlook.com>
ref tikv#12842 Make apply adaptive to reduce high tail latency. Signed-off-by: Jay Lee <BusyJayLee@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12842 - add water metrics - fix potential panic when destroying a peer - fix incorrect store size Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
* util: Fix incorrect memory capacity Signed-off-by: Wish <breezewish@outlook.com> * Fix lints Signed-off-by: Wish <breezewish@outlook.com> * Check capacity with /proc/meminfo Signed-off-by: Wish <breezewish@outlook.com> Signed-off-by: Wish <breezewish@outlook.com>
* hotfix kvproto for global config Signed-off-by: husharp <jinhao.hu@pingcap.com> * make format happy Signed-off-by: husharp <jinhao.hu@pingcap.com> Signed-off-by: husharp <jinhao.hu@pingcap.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#13730 Signed-off-by: glorv <glorvs@163.com>
…ock request (tikv#14037) close tikv#14038, close pingcap/tidb#40114 Fixes the problem that when handling repeated acquire_pessimistic_lock requests is recevied, should_not_exist is ignored. TiKV provides idempotency for these RPC requests, but for acquire_pessimistic_lock, it ignored the possibility that the client may expect a pessimistic_rollback between two acquire_pessimistic_lock request on the same key. In this case the second request may come from another statement and carries `should_not_exist` that wasn't set in the previously finished pessimistic lock request. If the first request successfully acquired the lock and the pessimistic_rollback failed, TiKV may return a sucessful response, making the client believe that the key doesn't exist before. In some rare cases, this has risk to cause data inconsistency. Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12842 This PR fixes several bugs and metrics: - Now waterfall timer will be reset in before_write, the goal is to solve the confusion that stall writes can pollute the whole waterfall metrics. - Perf context is changed not to be associated with engine instance. Perf context is thread local and instance independent under the hook. - Fix flushed index advance failure due to suspicious flush. - Support print long uncommitted logs and fix incorrect commit time Signed-off-by: Jay Lee <BusyJayLee@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12842 Move transaction related code to txn_ext.rs. Fix the bug that snapshot doesn't set term and extra_op. Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
ref tikv#12842 1) add snapshot apply metrics 2) disable bloomfilter for raftkv-v2 for now until a proper ratio is found 3) disable rocksdb write stall for raftkv-v2 until the tablet flow control is fully verified. Signed-off-by: Qi Xu <tonyxuqqi@outlook.com> Co-authored-by: Qi Xu <tonyxuqqi@outlook.com>
ref tikv#12876 fix witness raft log gc panic and refactor Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> Co-authored-by: Xinye Tao <xy.tao@outlook.com>
ref tikv#12999 copr: support handling keyspace request Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
ref tikv#13730 Support priority-based scheduling for the scheduler worker pool. Signed-off-by: Connor1996 <zbk602423539@gmail.com> Co-authored-by: Xinye Tao <xy.tao@outlook.com>
Signed-off-by: lijie <lijie@pingcap.com>
ref tikv#12842 Add some configurations for RocksDB filter enhancements Signed-off-by: tabokie <xy.tao@outlook.com>
…h importing data keys (tikv#14583) ref tikv#12842, ref tikv#14095, ref tikv#14097 support renaming encrypted dir (inefficiently) and batch importing data keys Signed-off-by: tabokie <xy.tao@outlook.com>
ref tikv#14614 Signed-off-by: 3pointer <luancheng@pingcap.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
close tikv#14581 store heartbeat will report sending/recving count to the pd . Signed-off-by: bufferflies <1045931706@qq.com>
ref tikv#14579 raftstore,server: add enable_v2_compatible_learner config The new config is added to clean up hard code tiflash check Signed-off-by: Neil Shen <overvenus@gmail.com>
…LUE_LEN` (tikv#14618) close tikv#14619 fix a bug with `process_old_collation_kv` function. related with tikv#11931, forget process `physical_table_id_column_cnt` in process_old_collation_kv function Signed-off-by: Jason Mo <mohangjie1995@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
…ore-v2 (tikv#14584) ref tikv#14579 enable raftstore-v1 apply tablet snapshot sent from raftstore-v2 Signed-off-by: Spade A <u6748471@anu.edu.au> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
…#14574) ref tikv#14547 raft: peers shouldn't hibernate incorrectly when one node fails Signed-off-by: qupeng <qupeng@pingcap.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
* raftstore-v2: prevent resolving store 0 Do not cache invaild peer otherwise it may send raft message to store 0 during region split. Signed-off-by: Neil Shen <overvenus@gmail.com> * address comments Signed-off-by: Neil Shen <overvenus@gmail.com> --------- Signed-off-by: Neil Shen <overvenus@gmail.com>
* done Signed-off-by: Spade A <u6748471@anu.edu.au> * add panic Signed-off-by: Spade A <u6748471@anu.edu.au> --------- Signed-off-by: Spade A <u6748471@anu.edu.au> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
Signed-off-by: tonyxuqqi <tonyxuqi@outlook.com>
…time (tikv#14530) ref tikv#14321 Add the apply log duration metrics. Signed-off-by: tonyxuqqi <tonyxuqi@outlook.com>
close tikv#14595 Make tiflash engine compatible with gc peer Signed-off-by: Neil Shen <overvenus@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
close tikv#14630 cdc: support filter lossy DDL changes. We don't need to send those changes downstream. Signed-off-by: hi-rustin <rustin.liu@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
close tikv#14609 Update the Azure SDK to latest version to support later developments. Signed-off-by: LykxSassinator <nkcs_lykx@hotmail.com>
close tikv#14224 Fix fd leak caused by continuous profiling Signed-off-by: tabokie <xy.tao@outlook.com>
… requests (tikv#14637) close tikv#14636, ref pingcap/tidb#42937 Makes TiKV support checking whether the lock is primary when handling check_txn_status. Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
Signed-off-by: tonyxuqqi <tonyxuqi@outlook.com>
… into get_engine_type_api
close tikv#14664 Fix stale read by correct updating peers Signed-off-by: Neil Shen <overvenus@gmail.com> Co-authored-by: tonyxuqqi <tonyxuqi@outlook.com>
close tikv#14658 Record the missing check_leader gRPC metrics. Signed-off-by: you06 <you1474600@gmail.com>
…size (tikv#14625) ref tikv#12842 1) optimize the load based split config based on region size 2) polish a log message when it cannot find a target peer of the message. Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ref tikv#14575 Support decode simple write request in v1 Signed-off-by: lidezhu <lidezhu@pingcap.com> Co-authored-by: Xinye Tao <xy.tao@outlook.com>
…ikv#14611) ref tikv#14528 Signed-off-by: cfzjywxk <lsswxrxr@163.com>
ref tikv#12842 support dynamically adjusting write buffer settings Signed-off-by: tabokie <xy.tao@outlook.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ref tikv#14575 Make snapshot_meta accessible Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is changed and how it works?
Issue Number: Ref tikv#12842
What's Changed:
Related changes
pingcap/docs/pingcap/docs-cn:Check List
Tests
Release note