[v23.2.x] CORE-1722: compression: Use preallocated decompression buffers for lz4 #17826
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
A configuration setting is added to disable lz4 buffer preallocation. (cherry picked from commit 3ec7f4e)
An object which holds a pair of preallocated buffers is added. The buffers are char arrays allocated at startup. State of allocation is maintained along with the buffers, to ensure correctness when using these buffers with malloc and free calls. The c api is introduced in the next commit. (cherry picked from commit cf44e60)
The custom allocator wraps the managed buffers into a c struct which is used by lz4 to route malloc and free calls. If the feature is disabled the struct is constructed such that calls are directly sent to malloc and free. (cherry picked from commit 9116875)
(cherry picked from commit edfba3d)
An optional block size parameter is added so that the compression op can set large block size, this enables testing the newly added preallocated buffers for large block size. (cherry picked from commit 5b3d7a3)
(cherry picked from commit 34860a9)
graphcareful
approved these changes
Apr 15, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport of PR #17385
Fixes #17816
Manual backport because of changes since v23.2.x and missing googletest support
Backports Required
Release Notes