receiver/prometheus: glue and complete staleness marking for disappearing metrics #3423
Conversation
odeke-em
force-pushed
the
receiver-prometheus-add-staleness-tracking+emit-staleness-markers
branch
2 times, most recently
from
1a1339f
to
6ab3223
Compare
|
The test failures were from an exported struct whose name was changed in the past 21 hours from "service.AppSettings" to "service.CollectorSettings" per #3268 |
odeke-em
force-pushed
the
receiver-prometheus-add-staleness-tracking+emit-staleness-markers
branch
from
6ab3223
to
4e062a8
Compare
|
Kindly /cc-ing @tigrannajaryan @bogdandrutu |
|
The receiver/prometheusreceiver TestEndToEnd test is quite rigid from 2 years ago, so that'll require me digging as I figure out what it does then retrofit with NaNs for values. |
|
The build is failing and there are merge conflicts. Please fix. |
odeke-em
force-pushed
the
receiver-prometheus-add-staleness-tracking+emit-staleness-markers
branch
2 times, most recently
from
bdf50eb
to
f5b3e98
Compare
|
This PR was marked stale due to lack of activity. It will be closed in 7 days. |
|
@odeke-em the tests for Windows, Unit tests and build are still failing. Can you please fix? |
odeke-em
force-pushed
the
receiver-prometheus-add-staleness-tracking+emit-staleness-markers
branch
3 times, most recently
from
e29db94
to
f0edaab
Compare
|
Status checks don't finish, can't merge. Will close/open to see if it helps. |
odeke-em
force-pushed
the
receiver-prometheus-add-staleness-tracking+emit-staleness-markers
branch
from
fef92c0
to
044efa0
Compare
odeke-em
force-pushed
the
receiver-prometheus-add-staleness-tracking+emit-staleness-markers
branch
from
044efa0
to
0f47d55
Compare
|
Thanks @tigrannajaryan and @rakyll! The Windows test is spurious but the other failure let me examine what's tripping it. Should shortly pass. |
This is to fix data race that existed a long time before this code but manifested when Prometheus concurrently modified pReceiver.
Yup it was a data race from code unrelated to this PR, that existed because Prometheus's code would modify pReceiver concurrently with pReceiver.Start being invoked. |
|
Alright @tigrannajaryan @rakyll @alolita now all the related code and tests pass 🎉! The contrib failure is unrelated to this change |
odeke-em
deleted the
receiver-prometheus-add-staleness-tracking+emit-staleness-markers
branch
|
Thank you all for the patient reviews and for your time! |
…disappearing metrics (open-telemetry#3423)" This reverts commit 8b79380.
…disappearing metrics (open-telemetry#3423)" This reverts commit 8b79380.
…disappearing metrics (open-telemetry#3423)" This reverts commit 8b79380.
…heus, rather than making our own (#3989) * Revert "receiver/prometheus: glue and complete staleness marking for disappearing metrics (#3423)" This reverts commit 8b79380. * Revert "receiver/prometheus: add store to track stale metrics (#3414)" This reverts commit cdc1634. * stop dropping staleness markers from prometheus, and fix tests * add staleness end to end test from #3423 * fix import grouping
…heus, rather than making our own (#3991) * Revert "receiver/prometheus: glue and complete staleness marking for disappearing metrics (#3423)" This reverts commit 8b79380. * Revert "receiver/prometheus: add store to track stale metrics (#3414)" This reverts commit cdc1634. * stop dropping staleness markers from prometheus, and fix tests * add staleness end to end test from #3423 * fix import grouping
…heus, rather than making our own (#3990) * Revert "receiver/prometheus: glue and complete staleness marking for disappearing metrics (#3423)" This reverts commit 8b79380. * Revert "receiver/prometheus: add store to track stale metrics (#3414)" This reverts commit cdc1634. * stop dropping staleness markers from prometheus, and fix tests * add staleness end to end test from #3423 * fix import grouping
Adds the staleness store to the metricsBuilder and transaction to track when metrics
disappear between scrapes.
Depends on #3414
Fixes #3413
Fixes #2961