Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make rpk generated dashboards work with "public_metrics" #5646

Closed
VladLazar opened this issue Jul 26, 2022 · 0 comments · Fixed by #6165
Closed

Make rpk generated dashboards work with "public_metrics" #5646

VladLazar opened this issue Jul 26, 2022 · 0 comments · Fixed by #6165
Labels
area/rpk kind/bug Something isn't working

Comments

@VladLazar
Copy link
Contributor

Context

This ticket is related to the "/public_metrics" Prometheus endpoint introduced in v22.2 Redpanda now exposes two Prometheus endpoints:

  • /metrics - "old" metrics endpoint, mostly unchanged but now has aggregated metrics which decreases the total number of metrics.
  • /public_metrics - "new" metrics endpoint, fewer metrics, better documented, but not complete yet for a monitoring solution.

For further info, see the PRD, the RFC for cardinality reduction and for the new endpoint.

The dashboards generated by rpk via rpk generate grafana-dashboard are slightly broken when using the "/public_metrics" endpoint.
We should update them, especially since we are going to make "/public_metrics" the default endpoint used for generate grafana-dashboards (PR here).

If you wish to see how the generated dashboards look like when using "public_metrics" currently, check out this dashboard.

Fixing the Dashboards

Currently, the generated dashboards use harcoded metrics from the "metrics" endpoint for the summary panels (code here). We should update any panels that use hardcoded metrics from the "metrics" endpoint (prefixed with vectorized) to use "public_metrics". Note that this may require coming up with a new PromQL query.

Folow-up

The scope of this issue is limited to making the currently generated dashboards work with the new endpoint. However, we should have a discussion at some point about what dashboards we want to provide to users out of the box and the delivery mechanism (rpk. download from our website, etc.)

Related Issues

#5501

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/rpk kind/bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants