Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove deprecated configurations #6673

Merged
merged 5 commits into from
Nov 17, 2023
Merged

Remove deprecated configurations #6673

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1d6581c
Remove -querier.query-ingesters-within config
leizor Nov 15, 2023
cc2cbda
Remove -querier.iterators and -querier.batch-iterators
leizor Nov 15, 2023
4eb6c90
Remove deprecated bucket store flags
leizor Nov 15, 2023
d75c63f
Remove -blocks-storage.bucket-store.bucket-index.enabled config
leizor Nov 15, 2023
94fc1cb
Update CHANGELOG.md
leizor Nov 15, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@

### Grafana Mimir

* [CHANGE] The following deprecated configurations have been removed: #6673
* `-querier.query-ingesters-within`
Copy link
Contributor

@ying-jeanne ying-jeanne Nov 16, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably aslo need to update runbook https://github.com/grafana/mimir/blob/main/docs/sources/mimir/manage/mimir-runbooks/_index.md. -querier.query-ingesters-within is still mentioned.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the flag still exists, but it's now a tenant-level override

* `-querier.iterators`
* `-querier.batch-iterators`
* `-blocks-storage.bucket-store.max-chunk-pool-bytes`
* `-blocks-storage.bucket-store.chunk-pool-min-bucket-size-bytes`
* `-blocks-storage.bucket-store.chunk-pool-max-bucket-size-bytes`
* `-blocks-storage.bucket-store.bucket-index.enabled`
* [CHANGE] Querier: Split worker GRPC config into separate client configs for the frontend and scheduler to allow TLS to be configured correctly when specifying the `tls_server_name`. The GRPC config specified under `-querier.frontend-client.*` will no longer apply to the scheduler client, and will need to be set explicitly under `-querier.scheduler-client.*`. #6445 #6573
* [CHANGE] Store-gateway: enable sparse index headers by default. Sparse index headers reduce the time to load an index header up to 90%. #6005
* [CHANGE] Store-gateway: lazy-loading concurrency limit default value is now 4. #6004
Expand Down
66 changes: 0 additions & 66 deletions cmd/mimir/config-descriptor.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1658,28 +1658,6 @@
"required": false,
"desc": "",
"blockEntries": [
{
"kind": "field",
"name": "iterators",
"required": false,
"desc": "Use iterators to execute query, as opposed to fully materialising the series in memory.",
"fieldValue": null,
"fieldDefaultValue": false,
"fieldFlag": "querier.iterators",
"fieldType": "boolean",
"fieldCategory": "deprecated"
},
{
"kind": "field",
"name": "batch_iterators",
"required": false,
"desc": "Use batch iterators to execute query, as opposed to fully materialising the series in memory. Takes precedent over the -querier.iterators flag.",
"fieldValue": null,
"fieldDefaultValue": true,
"fieldFlag": "querier.batch-iterators",
"fieldType": "boolean",
"fieldCategory": "deprecated"
},
{
"kind": "field",
"name": "query_store_after",
Expand Down Expand Up @@ -8081,17 +8059,6 @@
"required": false,
"desc": "",
"blockEntries": [
{
"kind": "field",
"name": "enabled",
"required": false,
"desc": "If enabled, queriers and store-gateways discover blocks by reading a bucket index (created and updated by the compactor) instead of periodically scanning the bucket.",
"fieldValue": null,
"fieldDefaultValue": true,
"fieldFlag": "blocks-storage.bucket-store.bucket-index.enabled",
"fieldType": "boolean",
"fieldCategory": "deprecated"
},
{
"kind": "field",
"name": "update_on_error_interval",
Expand Down Expand Up @@ -8140,39 +8107,6 @@
"fieldType": "duration",
"fieldCategory": "advanced"
},
{
"kind": "field",
"name": "max_chunk_pool_bytes",
"required": false,
"desc": "Max size - in bytes - of a chunks pool, used to reduce memory allocations. The pool is shared across all tenants. 0 to disable the limit.",
"fieldValue": null,
"fieldDefaultValue": 2147483648,
"fieldFlag": "blocks-storage.bucket-store.max-chunk-pool-bytes",
"fieldType": "int",
"fieldCategory": "deprecated"
},
{
"kind": "field",
"name": "chunk_pool_min_bucket_size_bytes",
"required": false,
"desc": "Size - in bytes - of the smallest chunks pool bucket.",
"fieldValue": null,
"fieldDefaultValue": 16000,
"fieldFlag": "blocks-storage.bucket-store.chunk-pool-min-bucket-size-bytes",
"fieldType": "int",
"fieldCategory": "deprecated"
},
{
"kind": "field",
"name": "chunk_pool_max_bucket_size_bytes",
"required": false,
"desc": "Size - in bytes - of the largest chunks pool bucket.",
"fieldValue": null,
"fieldDefaultValue": 50000000,
"fieldFlag": "blocks-storage.bucket-store.chunk-pool-max-bucket-size-bytes",
"fieldType": "int",
"fieldCategory": "deprecated"
},
{
"kind": "field",
"name": "series_hash_cache_max_size_bytes",
Expand Down
12 changes: 0 additions & 12 deletions cmd/mimir/help-all.txt.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -295,18 +295,12 @@ Usage of ./cmd/mimir/mimir:
This option controls how many series to fetch per batch. The batch size must be greater than 0. (default 5000)
-blocks-storage.bucket-store.block-sync-concurrency int
Maximum number of concurrent blocks synching per tenant. (default 20)
-blocks-storage.bucket-store.bucket-index.enabled
[deprecated] If enabled, queriers and store-gateways discover blocks by reading a bucket index (created and updated by the compactor) instead of periodically scanning the bucket. (default true)
-blocks-storage.bucket-store.bucket-index.idle-timeout duration
How long a unused bucket index should be cached. Once this timeout expires, the unused bucket index is removed from the in-memory cache. This option is used only by querier. (default 1h0m0s)
-blocks-storage.bucket-store.bucket-index.max-stale-period duration
The maximum allowed age of a bucket index (last updated) before queries start failing because the bucket index is too old. The bucket index is periodically updated by the compactor, and this check is enforced in the querier (at query time). (default 1h0m0s)
-blocks-storage.bucket-store.bucket-index.update-on-error-interval duration
How frequently a bucket index, which previously failed to load, should be tried to load again. This option is used only by querier. (default 1m0s)
-blocks-storage.bucket-store.chunk-pool-max-bucket-size-bytes int
[deprecated] Size - in bytes - of the largest chunks pool bucket. (default 50000000)
-blocks-storage.bucket-store.chunk-pool-min-bucket-size-bytes int
[deprecated] Size - in bytes - of the smallest chunks pool bucket. (default 16000)
-blocks-storage.bucket-store.chunks-cache.attributes-in-memory-max-items int
Maximum number of object attribute items to keep in a first level in-memory LRU cache. Metadata will be stored and fetched in-memory before hitting the cache backend. 0 to disable the in-memory cache. (default 50000)
-blocks-storage.bucket-store.chunks-cache.attributes-ttl duration
Expand Down Expand Up @@ -527,8 +521,6 @@ Usage of ./cmd/mimir/mimir:
[experimental] If enabled, store-gateway will persist a sparse version of the index-header to disk on construction and load sparse index-headers from disk instead of the whole index-header. (default true)
-blocks-storage.bucket-store.index-header.verify-on-load
If true, verify the checksum of index headers upon loading them (either on startup or lazily when lazy loading is enabled). Setting to true helps detect disk corruption at the cost of slowing down index header loading.
-blocks-storage.bucket-store.max-chunk-pool-bytes uint
[deprecated] Max size - in bytes - of a chunks pool, used to reduce memory allocations. The pool is shared across all tenants. 0 to disable the limit. (default 2147483648)
-blocks-storage.bucket-store.max-concurrent int
Max number of concurrent queries to execute against the long-term storage. The limit is shared across all tenants. (default 100)
-blocks-storage.bucket-store.meta-sync-concurrency int
Expand Down Expand Up @@ -1611,8 +1603,6 @@ Usage of ./cmd/mimir/mimir:
Minimum time to wait for ring stability at startup, if set to positive value. Set to 0 to disable.
-print.config
Print the config and exit.
-querier.batch-iterators
[deprecated] Use batch iterators to execute query, as opposed to fully materialising the series in memory. Takes precedent over the -querier.iterators flag. (default true)
-querier.cardinality-analysis-enabled
Enables endpoints used for cardinality analysis.
-querier.default-evaluation-interval duration
Expand Down Expand Up @@ -1667,8 +1657,6 @@ Usage of ./cmd/mimir/mimir:
Override the expected name on the server certificate.
-querier.id string
Querier ID, sent to the query-frontend to identify requests from the same querier. Defaults to hostname.
-querier.iterators
[deprecated] Use iterators to execute query, as opposed to fully materialising the series in memory.
-querier.label-names-and-values-results-max-size-bytes int
Maximum size in bytes of distinct label names and values. When querier receives response from ingester, it merges the response with responses from other ingesters. This maximum size limit is applied to the merged(distinct) results. If the limit is reached, an error is returned. (default 419430400)
-querier.label-values-max-cardinality-label-names-per-request int
Expand Down
30 changes: 0 additions & 30 deletions docs/sources/mimir/references/configuration-parameters/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1168,17 +1168,6 @@ instance_limits:
The `querier` block configures the querier.

```yaml
# (deprecated) Use iterators to execute query, as opposed to fully materialising
# the series in memory.
# CLI flag: -querier.iterators
[iterators: <boolean> | default = false]

# (deprecated) Use batch iterators to execute query, as opposed to fully
# materialising the series in memory. Takes precedent over the
# -querier.iterators flag.
# CLI flag: -querier.batch-iterators
[batch_iterators: <boolean> | default = true]

# (advanced) The time after which a metric should be queried from storage and
# not just ingesters. 0 means all queries are sent to store. If this option is
# enabled, the time range of the query sent to the store-gateway will be
Expand Down Expand Up @@ -3549,12 +3538,6 @@ bucket_store:
[ignore_deletion_mark_delay: <duration> | default = 1h]

bucket_index:
# (deprecated) If enabled, queriers and store-gateways discover blocks by
# reading a bucket index (created and updated by the compactor) instead of
# periodically scanning the bucket.
# CLI flag: -blocks-storage.bucket-store.bucket-index.enabled
[enabled: <boolean> | default = true]

# (advanced) How frequently a bucket index, which previously failed to load,
# should be tried to load again. This option is used only by querier.
# CLI flag: -blocks-storage.bucket-store.bucket-index.update-on-error-interval
Expand All @@ -3581,19 +3564,6 @@ bucket_store:
# CLI flag: -blocks-storage.bucket-store.ignore-blocks-within
[ignore_blocks_within: <duration> | default = 10h]

# (deprecated) Max size - in bytes - of a chunks pool, used to reduce memory
# allocations. The pool is shared across all tenants. 0 to disable the limit.
# CLI flag: -blocks-storage.bucket-store.max-chunk-pool-bytes
[max_chunk_pool_bytes: <int> | default = 2147483648]

# (deprecated) Size - in bytes - of the smallest chunks pool bucket.
# CLI flag: -blocks-storage.bucket-store.chunk-pool-min-bucket-size-bytes
[chunk_pool_min_bucket_size_bytes: <int> | default = 16000]

# (deprecated) Size - in bytes - of the largest chunks pool bucket.
# CLI flag: -blocks-storage.bucket-store.chunk-pool-max-bucket-size-bytes
[chunk_pool_max_bucket_size_bytes: <int> | default = 50000000]

# (advanced) Max size - in bytes - of the in-memory series hash cache. The
# cache is shared across all tenants and it's used only when query sharding is
# enabled.
Expand Down
4 changes: 4 additions & 0 deletions integration/backward_compatibility_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,10 @@ func runBackwardCompatibilityTest(t *testing.T, previousImage string, oldFlagsMa
// The distributor should have 512 tokens for the ingester ring and 1 for the distributor ring
require.NoError(t, distributor.WaitSumMetrics(e2e.Equals(512+1), "cortex_ring_tokens_total"))

// Start the compactor to have the bucket index created before querying.
compactor := e2emimir.NewCompactor("compactor", consul.NetworkHTTPEndpoint(), flags)
require.NoError(t, s.StartAndWaitReady(compactor))

checkQueries(t, consul, previousImage, flags, oldFlagsMapper, s, 1, instantQueryTest{
expr: "series_1",
time: series1Timestamp,
Expand Down
3 changes: 0 additions & 3 deletions integration/configs.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,6 @@ var (
BlocksStorageFlags = func() map[string]string {
return map[string]string{
"-blocks-storage.tsdb.block-ranges-period": "1m",
"-blocks-storage.bucket-store.bucket-index.enabled": "false",
"-blocks-storage.bucket-store.ignore-blocks-within": "0",
"-blocks-storage.bucket-store.sync-interval": "5s",
"-blocks-storage.tsdb.retention-period": "5m",
Expand Down Expand Up @@ -192,8 +191,6 @@ blocks_storage:

bucket_store:
sync_interval: 5s
bucket_index:
enabled: false

s3:
bucket_name: mimir-blocks
Expand Down
1 change: 0 additions & 1 deletion integration/getting_started_with_gossiped_ring_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,6 @@ func TestGettingStartedWithGossipedRing(t *testing.T) {
flags := map[string]string{
// decrease timeouts to make test faster. should still be fine with two instances only
"-ingester.ring.observe-period": "5s", // to avoid conflicts in tokens
"-blocks-storage.bucket-store.bucket-index.enabled": "false",
"-blocks-storage.bucket-store.sync-interval": "1s", // sync continuously
"-blocks-storage.bucket-store.ignore-blocks-within": "0",
"-blocks-storage.backend": "s3",
Expand Down
15 changes: 7 additions & 8 deletions integration/ingester_limits_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,14 +187,13 @@ overrides:
require.NoError(t, copyFileToSharedDir(s, "docs/configurations/single-process-config-blocks.yaml", mimirConfigFile))

flags := map[string]string{
"-runtime-config.reload-period": "100ms",
"-blocks-storage.backend": "filesystem",
"-blocks-storage.filesystem.dir": "/tmp",
"-blocks-storage.storage-prefix": "blocks",
"-blocks-storage.bucket-store.bucket-index.enabled": "false",
"-ruler-storage.backend": "filesystem",
"-ruler-storage.local.directory": "/tmp", // Avoid warning "unable to list rules".
"-runtime-config.file": filepath.Join(e2e.ContainerSharedDir, overridesFile),
"-runtime-config.reload-period": "100ms",
"-blocks-storage.backend": "filesystem",
"-blocks-storage.filesystem.dir": "/tmp",
"-blocks-storage.storage-prefix": "blocks",
"-ruler-storage.backend": "filesystem",
"-ruler-storage.local.directory": "/tmp", // Avoid warning "unable to list rules".
"-runtime-config.file": filepath.Join(e2e.ContainerSharedDir, overridesFile),
}
cortex1 := e2emimir.NewSingleBinary("cortex-1", flags, e2emimir.WithConfigFile(mimirConfigFile), e2emimir.WithPorts(9009, 9095))
require.NoError(t, s.StartAndWaitReady(cortex1))
Expand Down
Loading
Loading