Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ruler does not expose shipper metrics #2050

Closed
OGKevin opened this issue Jan 27, 2020 · 0 comments · Fixed by #2051
Closed

Ruler does not expose shipper metrics #2050

OGKevin opened this issue Jan 27, 2020 · 0 comments · Fixed by #2051

Comments

@OGKevin
Copy link
Contributor

OGKevin commented Jan 27, 2020

Thanos, Prometheus and Golang version used:

0.10.1

Object Storage Provider:
GCS

What happened:
Ruler failed to upload to GCS bucket, by reading the code, it should increase and expose the following metric:

if uploadErrs > 0 {
s.metrics.uploadFailures.Add(float64(uploadErrs))
return uploaded, errors.Errorf("failed to sync %v blocks", uploadErrs)

However, this does not happen due to missing prom register:

s := shipper.New(logger, nil, dataDir, bkt, func() labels.Labels { return lset }, metadata.RulerSource)

What you expected to happen:
Ruler to expose this metric.

How to reproduce it (as minimally and precisely as possible):

  1. run ruler with bad bucket config
  2. let the upload fail
  3. check if metric is exposed, this should not be the case

Full logs to relevant components:

Anything else we need to know:

OGKevin added a commit to OGKevin/thanos that referenced this issue Jan 27, 2020
OGKevin added a commit to OGKevin/thanos that referenced this issue Jan 28, 2020
thanos-io#2050
Fixes thanos-io#2050

Signed-off-by: Kevin Hellemun <17928966+OGKevin@users.noreply.github.com>
OGKevin added a commit to OGKevin/thanos that referenced this issue Jan 28, 2020
thanos-io#2050

Signed-off-by: Kevin Hellemun <17928966+OGKevin@users.noreply.github.com>
squat pushed a commit that referenced this issue Jan 28, 2020
* Pass prom registry to shipper from ruler.

#2050
Fixes #2050

Signed-off-by: Kevin Hellemun <17928966+OGKevin@users.noreply.github.com>

* Updated CHANGELOG.

#2050

Signed-off-by: Kevin Hellemun <17928966+OGKevin@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants