[pull] main from thanos-io:main #468
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
This commit splits the single promql_query_exec span into two separate spans, covering query creation and execution. Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
* receive: remove serverAsClient usage Remove serverAsClient usage to reduce CPU usage. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> * receive: remove unused param Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> * receive: make local client lazy Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> --------- Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
Split promql span into query create and exec spans
Previously we defered starting the gRPC server by blocking the whole startup until we could ping prometheus. This breaks usecases that rely on the config reloader to start prometheus. We fix it by using a channel to defer starting the grpc server and loading external labels in an actor concurrently. Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>
Dependency: Update minio-go to v7.0.70 which includes support for EKS Pod Identity. Signed-off-by: farhad <eqfarhad@gmail.com>
* Uupdate Prometheus Signed-off-by: alanprot <alanprot@gmail.com> * Updating prometheus to 4e664035e84e Signed-off-by: alanprot <alanprot@gmail.com> * Temporarily pinning prometheus common Signed-off-by: alanprot <alanprot@gmail.com> * fixing lint Signed-off-by: alanprot <alanprot@gmail.com> * Using jsoniter to encode promql responses Signed-off-by: alanprot <alanprot@gmail.com> * Removing e2e test case with unvalid hifen on a matcher -> prometheus now support this use case Signed-off-by: alanprot <alanprot@gmail.com> * Updating prometheus to v0.52.2-0.20240606174736-edd558884b24 Signed-off-by: alanprot <alanprot@gmail.com> * pinning grpc to v1.63.2 Signed-off-by: alanprot <alanprot@gmail.com> --------- Signed-off-by: alanprot <alanprot@gmail.com> Co-authored-by: EC2 Default User <ec2-user@ip-172-31-21-10.us-west-2.compute.internal>
Changelog - update the changelog entry position
The distributed engine retrieves label sets once per query, and doing the expensive copying and conversion uses a lot of memory. We already set them in the format we need in the endpoint status, so we can retrieve them from there. Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
Fetches the right version of prometheus from the releases api rather than the tags api Signed-off-by: Aritra24 <24430013+aritra24@users.noreply.github.com>
* Remove unused/broken `vendor` key. * Increase Go PR limit from 5 to 20. * Fixup yaml consistency. Signed-off-by: SuperQ <superq@gmail.com>
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2 to 3. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@v2...v3) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 3 to 6. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@v3...v6) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github.com/felixge/fgprof](https://github.com/felixge/fgprof) from 0.9.2 to 0.9.4. - [Release notes](https://github.com/felixge/fgprof/releases) - [Commits](felixge/fgprof@v0.9.2...v0.9.4) --- updated-dependencies: - dependency-name: github.com/felixge/fgprof dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress) from 1.17.8 to 1.17.9. - [Release notes](https://github.com/klauspost/compress/releases) - [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml) - [Commits](klauspost/compress@v1.17.8...v1.17.9) --- updated-dependencies: - dependency-name: github.com/klauspost/compress dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.29.0 to 1.33.1. - [Release notes](https://github.com/onsi/gomega/releases) - [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md) - [Commits](onsi/gomega@v1.29.0...v1.33.1) --- updated-dependencies: - dependency-name: github.com/onsi/gomega dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 3 to 5. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@v3...v5) --- updated-dependencies: - dependency-name: actions/setup-go dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Refactor existing stats incrementation for touched and fetched data Signed-off-by: Justin Jung <jungjust@amazon.com> * Add TypedBytesLimiter Signed-off-by: Justin Jung <jungjust@amazon.com> * Remove addAndCheck func Signed-off-by: Justin Jung <jungjust@amazon.com> * Update BytesLimiter interface to accept dataType param Signed-off-by: Justin Jung <jungjust@amazon.com> * Added tests Signed-off-by: Justin Jung <jungjust@amazon.com> * Fix build + changelog Signed-off-by: Justin Jung <jungjust@amazon.com> * Fix wrong data type Signed-off-by: Justin Jung <jungjust@amazon.com> * Changed storeDataType to be exported Signed-off-by: Justin Jung <jungjust@amazon.com> * Revert []BytesLimiter to BytesLimtier Signed-off-by: Justin Jung <jungjust@amazon.com> * Lint Signed-off-by: Justin Jung <jungjust@amazon.com> * More reverts Signed-off-by: Justin Jung <jungjust@amazon.com> * More Signed-off-by: Justin Jung <jungjust@amazon.com> * Rename DefaultBytesLimiterFactory back to NewBytesLimiterFactory Signed-off-by: Justin Jung <jungjust@amazon.com> * Changed StoreDataType from string to int Signed-off-by: Justin Jung <jungjust@amazon.com> * Removed nil check for bytesLimiter Signed-off-by: Justin Jung <jungjust@amazon.com> * nit Signed-off-by: Justin Jung <jungjust@amazon.com> * Removed changelog Signed-off-by: Justin Jung <jungjust@amazon.com> --------- Signed-off-by: Justin Jung <jungjust@amazon.com>
Dependabot already includes `dependencies` on PRs. Removing the config will cause it to also include an ecosystem label like `go`. Signed-off-by: SuperQ <superq@gmail.com>
Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com>
* upgrade Prometheus common and Prometheus to latest main Signed-off-by: Ben Ye <benye@amazon.com> * lint Signed-off-by: Ben Ye <benye@amazon.com> --------- Signed-off-by: Ben Ye <benye@amazon.com>
Signed-off-by: Aleksei Atavin <axeo@aiven.io>
The TSDB Selector is more powerful then label matchers. The issue is that we propagate the TSDB to select with label matchers, but they cannot convey enough information to select the right TSDB. This is an example of a configuration that would select too many TSDBs. Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>
Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>
Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>
Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>
Bumps [github.com/opentracing/basictracer-go](https://github.com/opentracing/basictracer-go) from 1.0.0 to 1.1.0. - [Release notes](https://github.com/opentracing/basictracer-go/releases) - [Commits](opentracing/basictracer-go@v1.0.0...v1.1.0) --- updated-dependencies: - dependency-name: github.com/opentracing/basictracer-go dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [go.elastic.co/apm/module/apmot](https://github.com/elastic/apm-agent-go) from 1.11.0 to 1.15.0. - [Release notes](https://github.com/elastic/apm-agent-go/releases) - [Changelog](https://github.com/elastic/apm-agent-go/blob/main/CHANGELOG.asciidoc) - [Commits](elastic/apm-agent-go@v1.11.0...v1.15.0) --- updated-dependencies: - dependency-name: go.elastic.co/apm/module/apmot dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [go.opentelemetry.io/otel/bridge/opentracing](https://github.com/open-telemetry/opentelemetry-go) from 1.28.0 to 1.29.0. - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.28.0...v1.29.0) --- updated-dependencies: - dependency-name: go.opentelemetry.io/otel/bridge/opentracing dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…7688) Bumps [go.opentelemetry.io/contrib/propagators/autoprop](https://github.com/open-telemetry/opentelemetry-go-contrib) from 0.53.0 to 0.54.0. - [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go-contrib@zpages/v0.53.0...zpages/v0.54.0) --- updated-dependencies: - dependency-name: go.opentelemetry.io/contrib/propagators/autoprop dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.26.5 to 3.26.6. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@2c779ab...4dd1613) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Saswata Mukherjee <saswataminsta@yahoo.com>
Co-authored-by: dominic.qi <dominic.qi@jaco.live> Co-authored-by: Ben Ye <benye@amazon.com>
* server/grpc: add pooling Add pooling for grpc requests/responses. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> * *: rm ZLabels and friends Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> * *: fix tests Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> * go.mod: revert changes Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> --------- Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
Do not double lock here as in some situations it could lead to a dead-lock situation. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
We support the Info gRPC api for 3 years now. We used to use Store API Info as fallback if we encounter an endpoint that does not implement Info gRPC but that should not happen now anymore. Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>
Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>
Removed all gogoproto extensions and dealt with the changes. 2nd step in removing gogoproto. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
Finally, let's enable vtprotobuf! Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
* query: add partition labels flag The distributed engine decides when to push down certain operations by checking if the external labels are still present, i.e. we can push down a binary operation if its vector matching includes all external labels. This is great but if you have multiple external labels that are irrelevant for the partition this is problematic since query authors must be aware of those irrelevant labels and must incorporate them into their queries. This PR attempts to solve that by giving an option to focus on the labels that are relevant for the partition. Signed-off-by: Michael Hoffmann <mhoffm@posteo.de> * Update cmd/thanos/query.go Co-authored-by: Filip Petkovski <filip.petkovsky@gmail.com> Signed-off-by: Michael Hoffmann <mhoffm@posteo.de> --------- Signed-off-by: Michael Hoffmann <mhoffm@posteo.de> Co-authored-by: Filip Petkovski <filip.petkovsky@gmail.com>
* store: added a log error print when proxy limit are violated Signed-off-by: Tidhar Klein Orbach <tidhar.o@taboola.com> * Update pkg/store/proxy.go Co-authored-by: Pedro Tanaka <pedro.tanaka@shopify.com> Signed-off-by: Tidhar Klein Orbach <tizki@users.noreply.github.com> --------- Signed-off-by: Tidhar Klein Orbach <tidhar.o@taboola.com> Signed-off-by: Tidhar Klein Orbach <tizki@users.noreply.github.com> Co-authored-by: Pedro Tanaka <pedro.tanaka@shopify.com>
Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>
Use the new CodecV2 interface to enable pooling gRPC marshaling/unmarshaling buffers. Also, add missing includes to scripts/genproto.sh so that we could enable the `pool` flag in the next PR. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>
Now that we have generics, we can generalize the bucketed bytes pool to be used with slices of any type T. Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>
* part_1 Signed-off-by: Thibault Mange <22740367+thibaultmg@users.noreply.github.com> * typo Signed-off-by: Thibault Mange <22740367+thibaultmg@users.noreply.github.com> * Update docs/blog/2023-11-20-life-of-a-sample-part-1.md Co-authored-by: Pedro Tanaka <pedro.tanaka@shopify.com> Signed-off-by: Thibault Mange <22740367+thibaultmg@users.noreply.github.com> * Update docs/blog/2023-11-20-life-of-a-sample-part-1.md Co-authored-by: Pedro Tanaka <pedro.tanaka@shopify.com> Signed-off-by: Thibault Mange <22740367+thibaultmg@users.noreply.github.com> * Update docs/blog/2023-11-20-life-of-a-sample-part-1.md Co-authored-by: Pedro Tanaka <pedro.tanaka@shopify.com> Signed-off-by: Thibault Mange <22740367+thibaultmg@users.noreply.github.com> * Update docs/blog/2023-11-20-life-of-a-sample-part-1.md Co-authored-by: Pedro Tanaka <pedro.tanaka@shopify.com> Signed-off-by: Thibault Mange <22740367+thibaultmg@users.noreply.github.com> * Update docs/blog/2023-11-20-life-of-a-sample-part-1.md Co-authored-by: Pedro Tanaka <pedro.tanaka@shopify.com> Signed-off-by: Thibault Mange <22740367+thibaultmg@users.noreply.github.com> * add sidecar, remove invalid links Signed-off-by: Thibault Mange <22740367+thibaultmg@users.noreply.github.com> --------- Signed-off-by: Thibault Mange <22740367+thibaultmg@users.noreply.github.com> Co-authored-by: Pedro Tanaka <pedro.tanaka@shopify.com>
Generalize the bucketed bytes pool
* add img style attribute Signed-off-by: Thibault Mange <22740367+thibaultmg@users.noreply.github.com> * fix formatting Signed-off-by: Thibault Mange <22740367+thibaultmg@users.noreply.github.com> * fix link Signed-off-by: Thibault Mange <22740367+thibaultmg@users.noreply.github.com> * remove internal links Signed-off-by: Thibault Mange <22740367+thibaultmg@users.noreply.github.com> --------- Signed-off-by: Thibault Mange <22740367+thibaultmg@users.noreply.github.com>
Just compare labelpb.Label directly instead of creating promlabels from them. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>
Do not create intermediate prometheus labels and compare the labels directly. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>
We use the stringlabels call so some allocations are inevitable but we
can be much smarter about it:
```
func (s *storeSeriesSet) At() (labels.Labels, []*storepb.AggrChunk) {
return s.series[s.i].PromLabels(), s.series[s.i].Chunks <--- not memoized, new alloc on every At() call; need to memoize because of stringlabel. One alloc is inevitable.
}
```
```
lset, chks := s.SeriesSet.At()
if s.peek == nil {
s.peek = &Series{Labels: labelpb.PromLabelsToLabelpbLabels(lset), Chunks: chks} <-- converting back to labelpb ?
continue
}
```
```
if labels.Compare(lset, s.peek.PromLabels()) != 0 { <--- PromLabels() called; we can avoid this call
s.lset, s.chunks = s.peek.PromLabels(), s.peek.Chunks <- PromLabels() called; we can avoid this
s.peek = &Series{Labels: labelpb.PromLabelsToLabelpbLabels(lset), Chunks: chks} <--- converting back to labelpb; we can avoid this
return true
}
```
Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
Website: remove ThanosCon banner
`main` has some breakage so use older version for now. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )