Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

merge master into features/rules-proxy #2623

Merged
merged 198 commits into from
May 20, 2020

Conversation

s-urbaniak
Copy link
Contributor

This is a merge commit to get all the changes from master into the feature branch so we can iterate.

bwplotka and others added 30 commits February 28, 2020 14:07
…2199)

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Corrected all Prometheus possessives to read `Prometheus's`, this matches Prometheus's own documentation.
* Corrected `simple` to `simply` when describing compactor scanning behaviour

Signed-off-by: Peter Avdjian <pavdjian@paperlesspost.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Roman Grytskiv <roman.grytskiv@gmail.com>
* Skip deleting files that we just deleted

We see this happening with Swift. Because the consistency of swift is eventual, swift sometimes didn't process the deletion of the meta file yet, and so it turns up in the bkt.Iter(). The second deletion then causes a 404 and compaction fails.

Signed-off-by: Wim Fournier <github@fournier.nl>

* return, as this is a func. Add debug log and comment

Signed-off-by: Wim Fournier <github@fournier.nl>

* fixing build: wrong parameter name

Signed-off-by: Wim Fournier <github@fournier.nl>

* fix lint

Signed-off-by: Wim Fournier <github@fournier.nl>

* Refactor deleteDir into deleteDirRec and add a parameter for a function that allows to keep certain files.

Signed-off-by: Wim Fournier <github@fournier.nl>

* Fix lint

Signed-off-by: Wim Fournier <github@fournier.nl>

* implementing suggested fixes

Signed-off-by: Wim Fournier <github@fournier.nl>
This makes the handling of web.route-prefix more similar to the
behavior in Prometheus.  Correctly handles '/' and prefixes which
do not begin with a '/'.

Signed-off-by: Paul Gier <pgier@redhat.com>
* Create release v0.11.0-rc.0 (thanos-io#2156)

* Update version to v0.11.0-rc.0

* Update CHANGELOG with all PRs for v0.11

* Improve CHANGELOG by being more explicit

* Bumped minio-go library to v6.0.49, fixing an IAM bug in v6.0.45 (thanos-io#2189)

Signed-off-by: Kraig Amador <kraig.amador@ticketmaster.com>

* Create release candidate  v0.11.0-rc.1 (thanos-io#2192)

Signed-off-by: Matthias Loibl <mail@matthiasloibl.com>

* Release v0.11.0 (thanos-io#2205)

Signed-off-by: Matthias Loibl <mail@matthiasloibl.com>

* Update VERSION to 0.12.0-dev

Signed-off-by: Matthias Loibl <mail@matthiasloibl.com>

* Resolve go.sum merge conflict and run go mod tidy

Signed-off-by: Matthias Loibl <mail@matthiasloibl.com>

Co-authored-by: Kraig Amador <508403+bigkraig@users.noreply.github.com>
* returns error messages when trigger reload with http

Signed-off-by: arthur yang <yang_yapo@126.com>

* use simple reloadRules function instead of magic chan error error

Signed-off-by: yapo.yang <yang_yapo@126.com>

* add tailing period for comment

Signed-off-by: yapo.yang <yang_yapo@126.com>

* fix comment

Signed-off-by: arthur yang <yang_yapo@126.com>

* add white space for better code reading

Signed-off-by: arthur yang <yang_yapo@126.com>

* collect thanos rule metrics into one struct

Signed-off-by: arthur yang <yang_yapo@126.com>

* remove termination logic and keep log only

Signed-off-by: arthur yang <yang_yapo@126.com>

* update changelog for thanos-io#1848

Signed-off-by: arthur yang <yang_yapo@126.com>

* add tailing period

Signed-off-by: arthur yang <yang_yapo@126.com>

* check whether registry is nil

Signed-off-by: arthur yang <yang_yapo@126.com>

* tailing period in metrics

Signed-off-by: arthur yang <yang_yapo@126.com>

* cancel with context

Signed-off-by: arthur yang <yang_yapo@126.com>

* return ctx.Err() instead of errors.New

Signed-off-by: arthur yang <yang_yapo@126.com>

* register thanos rule metrics with promauto

Signed-off-by: arthur yang <yang_yapo@126.com>

* return errs before set success related metrics

Signed-off-by: arthur yang <yang_yapo@126.com>

* revert go.sum go.mod change

Signed-off-by: arthur yang <yang_yapo@126.com>

* reload webhandler/sighup in one for loop

Signed-off-by: arthur yang <yang_yapo@126.com>

* reload with chan chan error

Signed-off-by: yapo.yang <yang_yapo@126.com>
Signed-off-by: mcsammac

Date:      Wed Mar 4 13:50:17 2020 -0500
On branch master
Changes to be committed:
	modified:   pkg/prober/intrumentation.go

Signed-off-by: s320009 <sam.mcadams@8451.com>
Signed-off-by: Paul Gier <pgier@redhat.com>
…ed unnecessary printfs. (thanos-io#2228)

Fixes: thanos-io#2102

Also blocked them on CI side, thanks to fatih/faillint#8

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
… full queue and dropped alerts (thanos-io#2238)

* Add test for alert queue Pop after multiple Push

Signed-off-by: Robin Clarke-Williams <43950815+robincw-gr@users.noreply.github.com>

* Fix alert queue bug by resignal after Pop (thanos-io#2204)

Signed-off-by: Robin Clarke-Williams <43950815+robincw-gr@users.noreply.github.com>

* Fix alert queue test and simplify

Signed-off-by: Robin Clarke-Williams <43950815+robincw-gr@users.noreply.github.com>

* Update CHANGELOG.md

Signed-off-by: Robin Clarke-Williams <43950815+robincw-gr@users.noreply.github.com>

* Link to thanos-io/thanos PR in CHANGELOG.md

Signed-off-by: Robin Clarke-Williams <43950815+robincw-gr@users.noreply.github.com>
Signed-off-by: Jacob Colvin <Jacob.Colvin@8451.com>
…2229)

Signed-off-by: Rajesh Rajendran <rjshrjndrn@gmail.com>
* Fix pkg/errors error directive issues

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Fix misuse of Errorf

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Signed-off-by: gitlawr <lawrleegle@gmail.com>
…os-io#2231)

Signed-off-by: Paul Traylor <paul.traylor@linecorp.com>
Signed-off-by: Richard Poole <richard.poole@cudoventures.com>
You would have to query `prometheus_tsdb_head_series` instead of `sum(prometheus_tsdb_head_series)` in order to get the 5 results when deduplicating.

Signed-off-by: John Chen <johnchen456@gmail.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* docs: Document Thanos Sharding

Signed-off-by: Xiang Dai <764524258@qq.com>

* Add time partitioning

Signed-off-by: Xiang Dai <764524258@qq.com>

* feedback

Signed-off-by: Xiang Dai <764524258@qq.com>
…kgroud (thanos-io#2272)

* Sharding: document supported relabel action and add store gateway background

Signed-off-by: Xiang Dai <764524258@qq.com>

* add hashmod

Signed-off-by: Xiang Dai <764524258@qq.com>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
…ls conversion. (thanos-io#2230)

## Changes

* method TranslateLables CPU Optimized (streamed sorting).
* All store GW label conversation to []storepb.Label are now alloc-less.

```
go test -bench=BenchmarkUnsafeVSSafeLabelsConversion -run=^$ -benchmem -timeout 2h -benchtime 10s ./pkg/store/storepb/...
 goos: linux
 goarch: amd64
 pkg: github.com/thanos-io/thanos/pkg/store/storepb
 BenchmarkUnsafeVSSafeLabelsConversion/safe-12         	   34822	    339076 ns/op	  655368 B/op	       2 allocs/op
 BenchmarkUnsafeVSSafeLabelsConversion/unsafe-12       	1000000000	         2.32 ns/op	       0 B/op	       0 allocs/op
PASS
```

TODO: Do the same on Querier.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: kushthedude <kushthedude@gmail.com>
Signed-off-by: Xiang Dai <764524258@qq.com>
Signed-off-by: yeya24 <yb532204897@gmail.com>
* scripts/genflagdocs.sh: Generate downsample flag

Signed-off-by: Xiang Dai <764524258@qq.com>

* Document downsample component

Signed-off-by: Xiang Dai <764524258@qq.com>

* Move downsample as bucket sub-command

Signed-off-by: Xiang Dai <764524258@qq.com>

* update docs

Signed-off-by: Xiang Dai <764524258@qq.com>

* feedback

Signed-off-by: Xiang Dai <764524258@qq.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
pstibrany and others added 21 commits May 7, 2020 17:39
* Added generic cache interface.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Added memcached implementation of Cache.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Chunks-caching bucket.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Fix sentences

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Fix sentences

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Fix sentences

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Rename config objects.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Review feedback.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Review feedback.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Added metrics for object size.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Added requested chunk bytes metric.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Caching bucket docs.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Fixed tests.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Fix test.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Update docs/components/store.md
Update pkg/store/cache/caching_bucket.go

Co-authored-by: Marco Pracucci <marco@pracucci.com>
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Dots

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Always set lastBlockOffset.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Merged cached metric into fetched metric, added labels.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Added CHANGELOG.md entry

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Reworded help for thanos_store_bucket_cache_fetched_chunk_bytes_total

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Added tracing around getRangeChunkFile method.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Updated CHANGELOG.md

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Options

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Fix parameter name. (store. got dropped by accident)

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Use embedded Bucket

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Added comments.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Fixed comment.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Hide store.caching-bucket.config flags.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Renamed block to subrange.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Renamed block to subrange.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Header

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Added TODO

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Removed TODO, in favor of creating issue.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Use NopCloser.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

Co-authored-by: Marco Pracucci <marco@pracucci.com>
Signed-off-by: Marco Pracucci <marco@pracucci.com>
Changes mainly made for consistency, like section headers being in imperative tense: "do this thing" instead of "this is the thing"

Signed-off-by: Stephen Weber <stephen.t.weber@gmail.com>
The subcommand is called now `tools bucket web` after the recent
changes.

Without this, the quickstart script outputs:
```
Error parsing commandline arguments: expected command but got "bucket"
thanos: error: expected command but got "bucket"
```

Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>
Signed-off-by: Joel Bastos <kintoandar@gmail.com>
Signed-off-by: Philip Gough <philip.p.gough@gmail.com>
* ui: add React UI from upstream Prometheus

Signed-off-by: Adrien Fillon <adrien.fillon@gmail.com>

* ui: incorporate new changes from Prometheus React UI

Signed-off-by: Prem Kumar <prmsrswt@gmail.com>

* ui: adapted the React UI to Thanos

Signed-off-by: Prem Kumar <prmsrswt@gmail.com>

Co-authored-by: Adrien Fillon <adrien.fillon@gmail.com>
Co-authored-by: Giedrius Statkevičius <giedriuswork@gmail.com>
Signed-off-by: Pierre-Yves Aillet <pyaillet@gmail.com>
* react: graph/panel: revert changes temporarily

Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>

* react-app: apply 'Update React vendoring'

Add the commit
prometheus/prometheus@65a1942
via a 3-way merge.

Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>

* ui/react-app: update yarn deps

Should fix security warnings. Ported from
prometheus/prometheus@24ecae9.

Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>

* ui: update bindata

Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>
Found out that there is some weird interaction between `jest --coverage`
and `babel-plugin-istanbul`. Maybe related to:
jestjs/jest#6827.

From my testing, removing `--coverage` makes this work again. Probably
worth investigating in the future why that happens.

Also, this is really not needed during CI because we do not use the
coverage data anywhere anyway.

Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>
* ci: test React UI using GitHub actions

Signed-off-by: Prem Kumar <prmsrswt@gmail.com>

* ci: remove react-app-test from CircleCI as we now use GH Actions

Signed-off-by: Prem Kumar <prmsrswt@gmail.com>
Signed-off-by: Lucas Servén Marín <lserven@gmail.com>
Unfortunately, all passes. ):

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
* Added caching for Iter.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Added cache for Exists call for meta-files.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Added cache for reading block metadata files.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Make caching bucket configurable with different caches for different type of objects.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Fixed tests.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Added caching for ObjectSize. Enabled caching of index.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Lint feedback.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Use single set of metrics for all operations.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Constants.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Use operation specific config. Generic configuration is only for user.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Fix typo, make lint happy.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Simplify constants.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Simplify caching configuration.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Refactor cache configuration.

Configuration is now passed to the cache when created.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Review feedback.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Fix operationRequests and operationHits for getRange.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Make codec for Iter results configurable.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Added header.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Renamed "dir" config to "blocks-iter".

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Bump default values for meta exists/doesntExist ttls.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Removed example how cache could be configured for index.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Address review feedback.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Get now implements streaming reader, and buffers object in memory.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Added test for partial read.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Removed unused function.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
* Update store.go

Signed-off-by: Harshitha Chowdary <harshithachowdary.t17@iiits.in>

* Update store.go

Signed-off-by: Harshitha Chowdary <harshithachowdary.t17@iiits.in>

* Update cmd/thanos/store.go

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Signed-off-by: Harshitha Chowdary <harshithachowdary.t17@iiits.in>

* Updated the docs folder

Signed-off-by: Harshitha Chowdary <harshithachowdary.t17@iiits.in>

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Co-authored-by: johncming <johncming@yahoo.com>
Signed-off-by: Lili Cosic <cosiclili@gmail.com>

Co-authored-by: johncming <johncming@yahoo.com>
* mixin/thanos: align mixin schema with other mixins

- promote default.libsonnet to main config file
- include config.libsonnet in mixin.libsonnet
- include jsonnetfile.json
- allow generating directly from mixin/thanos directory

Signed-off-by: paulfantom <pawel@krupa.net.pl>

* mixin: move one directory higher

Signed-off-by: paulfantom <pawel@krupa.net.pl>
)

* Documentation and CHANGELOG.md entry for thanos-io#2579.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Document hidden option to configure caching bucket, to make it less hidden.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Removed mention of caching_config.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Added periods.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
…ates. (thanos-io#2548)

* querier: Adjust deduplication for counters when querying for PromQL rates.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Added CHANGELOG.md

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Improved description as suggested by Frederic.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Moved err adjust to be per replica, not per already deduped value.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
@s-urbaniak
Copy link
Contributor Author

cc @bwplotka

@s-urbaniak s-urbaniak force-pushed the features/rules-proxy branch 2 times, most recently from 2df82c7 to c9f25f7 Compare May 18, 2020 13:41
Signed-off-by: Sergiusz Urbaniak <sergiusz.urbaniak@gmail.com>
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
@bwplotka bwplotka merged commit d6a5cc1 into thanos-io:features/rules-proxy May 20, 2020
@s-urbaniak s-urbaniak deleted the features/rules-proxy branch May 20, 2020 11:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.