docs: add note about PYTHONMALLOC for accurate jemalloc memory tracking #17709
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
Added a note in the documentation suggesting that users may set `PYTHONMALLOC=malloc` when using `jemalloc`. This allows jemalloc to track memory usage more accurately by bypassing Python's internal small-object allocator (`pymalloc`), helping to ensure that cache_autotuning functions as expected. This doc change aims to provide more clarity for users configuring jemalloc with Synapse.
hensg
force-pushed
the
docs-jemalloc
branch
from
ccc3603
to
894a811
Compare
I tried this setting with Docker Compose. Memory usage did go down, yet became much more erratic. On the following graph, the switching point is around 14h00.
|
|
@daedric7 could you share the version without malloc/jemalloc and your configurations? |
Almost 24h of memory usage before the change.
What other info can i provide ? |
|
Are you using |
As far as i was aware, jemalloc2 it's always and use when using Docker. Is this not true ? |
I checked start.py inside my container: Can you please confirm that jemalloc2 is not used by default with docker ? Everything else points to the oposite: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added a note in the documentation suggesting that users may set
PYTHONMALLOC=mallocwhen usingjemalloc. This allows jemalloc to track memory usage more accurately by bypassing Python's internal small-object allocator (pymalloc), helping to ensure thatcache_autotuningfunctions as expected.This doc change aims to provide more clarity for users configuring jemalloc with Synapse.
Based on:
synapse/synapse/metrics/jemalloc.py
Lines 198 to 201 in 4ac7835