Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix cortex_ingester_memory_series metric tracking at startup #4312

Merged
merged 2 commits into from
Feb 27, 2023

Conversation

pracucci
Copy link
Collaborator

@pracucci pracucci commented Feb 27, 2023

What this PR does

This PR fixes #4310 delegating cortex_ingester_memory_series metrics tracking to TSDB. The reason why it fixes #4310 is because we don't track the per-TSDB registerer until after tsdb.Open() returned, so the spike during the WAL replay is not accounted. This also makes cortex_ingester_memory_series consistent with cortex_ingester_memory_series_created_total and cortex_ingester_memory_series_removed_total metrics.

Which issue(s) this PR fixes or relates to

Fixes #4310

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci pracucci marked this pull request as ready for review February 27, 2023 11:35
@pracucci pracucci requested a review from a team as a code owner February 27, 2023 11:35
Copy link
Contributor

@aknuds1 aknuds1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see question.

CHANGELOG.md Outdated Show resolved Hide resolved
pkg/ingester/metrics.go Show resolved Hide resolved
@aknuds1 aknuds1 requested a review from a team February 27, 2023 12:42
Copy link
Contributor

@aknuds1 aknuds1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks sensible to me.

@aknuds1 aknuds1 requested a review from a team February 27, 2023 13:06
Copy link
Member

@pstibrany pstibrany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, lgtm.

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
@pracucci pracucci enabled auto-merge (squash) February 27, 2023 13:44
@pracucci pracucci merged commit b352388 into main Feb 27, 2023
@pracucci pracucci deleted the fix-memory-series-tracking branch February 27, 2023 13:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Invalid cortex_ingester_memory_series tracked during ingester startup
3 participants