Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

store: add ability to limit max num of samples / concurrent queries #798

Merged
merged 57 commits into from
Mar 23, 2019
Merged

store: add ability to limit max num of samples / concurrent queries #798

merged 57 commits into from
Mar 23, 2019

Commits on Feb 4, 2019

  1. store: add ability to limit max samples / conc. queries

    @GiedriusS
    Giedrius Statkevičius authored and GiedriusS committed Feb 4, 2019
    Configuration menu
    Copy the full SHA
    1bc1f59 View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2019

  1. store/bucket: account for the RawChunk case 

    Convert raw chunks into XOR encoded chunks and call the NumSamples()
method on them to calculate the number of samples. Rip out the samples
calculation into a different function because it is used in two
different places.
    Giedrius Statkevičius committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    e87f763 View commit details
    Browse the repository at this point in the history

  2. store/bucket_e2e_test: adjust sample limit size 

    It should be actually 30 - I miscalculated this.
    Giedrius Statkevičius committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    1ab1dc6 View commit details
    Browse the repository at this point in the history

  3. store/bucket: add metric thanos_bucket_store_queries_limited_total

    Giedrius Statkevičius committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    d7c3ade View commit details
    Browse the repository at this point in the history

  4. store/bucket: register queriesLimited metric

    Giedrius Statkevičius committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    12db24a View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2019

  1. store: make changes according to the review comments

    Giedrius Statkevičius committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    9d0b8a7 View commit details
    Browse the repository at this point in the history

  2. docs/store: update

    Giedrius Statkevičius committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    9727072 View commit details
    Browse the repository at this point in the history

  3. store: gating naming changes, add span/extra metric

    Giedrius Statkevičius committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    d9c733a View commit details
    Browse the repository at this point in the history

  4. store: improve error messages

    Giedrius Statkevičius committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    c4ce735 View commit details
    Browse the repository at this point in the history

  5. store/limiter: improve error messages

    Giedrius Statkevičius committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    30eef19 View commit details
    Browse the repository at this point in the history

  6. store/gate: time -> seconds

    Giedrius Statkevičius committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    194394d View commit details
    Browse the repository at this point in the history

  7. store/bucket_e2e_test: narrow down the first query

    Giedrius Statkevičius committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    2e51c2e View commit details
    Browse the repository at this point in the history

  8. store/bucket: check for negative maxConcurrent

    Giedrius Statkevičius committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    58a14fa View commit details
    Browse the repository at this point in the history

  9. cmd/store: clarify help message

    Giedrius Statkevičius committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    4d1b7ed View commit details
    Browse the repository at this point in the history

  10. Merge remote-tracking branch 'origin/master' into smpl

    Giedrius Statkevičius committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    3ae3733 View commit details
    Browse the repository at this point in the history

  11. pkg/store: hook thanos_bucket_store_queries_limited into Limiter

    Giedrius Statkevičius committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    b149f74 View commit details
    Browse the repository at this point in the history

  12. store/bucket_test: fix NewBucketStore call

    Giedrius Statkevičius committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    e79c56d View commit details
    Browse the repository at this point in the history

  13. docs: update again

    Giedrius Statkevičius committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    1d07515 View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2019

  1. store/gate: spelling fix

    Giedrius Statkevičius committed Feb 9, 2019
    Configuration menu
    Copy the full SHA
    38a093b View commit details
    Browse the repository at this point in the history

  2. store/gate: spelling fix #2

    Giedrius Statkevičius committed Feb 9, 2019
    Configuration menu
    Copy the full SHA
    7b13f7e View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2019

  1. Merge remote-tracking branch 'origin/master' into feature/store_sampl… 

    …e_limit
    @GiedriusS
    GiedriusS committed Feb 15, 2019
    Configuration menu
    Copy the full SHA
    4540394 View commit details
    Browse the repository at this point in the history

  2. store/bucket: remove pointless newline

    @GiedriusS
    GiedriusS committed Feb 15, 2019
    Configuration menu
    Copy the full SHA
    3e532fe View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2019

  1. store/gate: generalize gate timing 

    Make the metric show in general how much time it takes for queries to
wait at the gate.
    Giedrius Statkevičius committed Mar 14, 2019
    Configuration menu
    Copy the full SHA
    cff979c View commit details
    Browse the repository at this point in the history

  2. store/gate: convert the g.gateTiming metric into a histogram

    Giedrius Statkevičius committed Mar 14, 2019
    Configuration menu
    Copy the full SHA
    e7ea64b View commit details
    Browse the repository at this point in the history

  3. store/bucket: change comment wording

    Giedrius Statkevičius committed Mar 14, 2019
    Configuration menu
    Copy the full SHA
    23c7368 View commit details
    Browse the repository at this point in the history

  4. store/bucket: remove type from maxSamplesPerChunk 

    Let Go decide by itself what kind of type it needs.
    Giedrius Statkevičius committed Mar 14, 2019
    Configuration menu
    Copy the full SHA
    da575e4 View commit details
    Browse the repository at this point in the history

  5. store/bucket: rename metric into thanos_bucket_store_queries_dropped

    Giedrius Statkevičius committed Mar 14, 2019
    Configuration menu
    Copy the full SHA
    e390846 View commit details
    Browse the repository at this point in the history

  6. thanos/store: clarify help message 

    Literally explain what it means in the help message so that it would be
clearer.
    Giedrius Statkevičius committed Mar 14, 2019
    Configuration menu
    Copy the full SHA
    24e8e1f View commit details
    Browse the repository at this point in the history

  7. store/gate: rename metric to thanos_bucket_store_queries_in_flight 

    More fitting as decided by everyone.
    Giedrius Statkevičius committed Mar 14, 2019
    Configuration menu
    Copy the full SHA
    4012eca View commit details
    Browse the repository at this point in the history

  8. store/gate: fix MustRegister() call

    Giedrius Statkevičius committed Mar 14, 2019
    Configuration menu
    Copy the full SHA
    e7be55d View commit details
    Browse the repository at this point in the history

  9. docs: update

    Giedrius Statkevičius committed Mar 14, 2019
    Configuration menu
    Copy the full SHA
    3e8150d View commit details
    Browse the repository at this point in the history

  10. store/bucket: clarify the name of the span 

    Make it more clearer about what it is for.
    Giedrius Statkevičius committed Mar 14, 2019
    Configuration menu
    Copy the full SHA
    5ec5ce9 View commit details
    Browse the repository at this point in the history

  11. store/bucket: inline calculation into the function call 

    No need to create an extra variable in a hot path in the code if we can
inline it and it will be just as clear.
    Giedrius Statkevičius committed Mar 14, 2019
    Configuration menu
    Copy the full SHA
    810a131 View commit details
    Browse the repository at this point in the history

  12. Merge remote-tracking branch 'origin' into fork_store_sample_limit

    Giedrius Statkevičius committed Mar 14, 2019
    Configuration menu
    Copy the full SHA
    541f180 View commit details
    Browse the repository at this point in the history

  13. CHANGELOG: add item about this

    Giedrius Statkevičius committed Mar 14, 2019
    Configuration menu
    Copy the full SHA
    ae8e425 View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2019

  1. store/gate: reduce number of buckets

    Giedrius Statkevičius committed Mar 15, 2019
    Configuration menu
    Copy the full SHA
    de8a234 View commit details
    Browse the repository at this point in the history

  2. store/bucket: rename metric to thanos_bucket_store_queries_dropped_total

    Giedrius Statkevičius committed Mar 15, 2019
    Configuration menu
    Copy the full SHA
    07b4658 View commit details
    Browse the repository at this point in the history

  3. store/bucket: move defer out of code block

    Giedrius Statkevičius committed Mar 15, 2019
    Configuration menu
    Copy the full SHA
    61d6ecd View commit details
    Browse the repository at this point in the history

  4. store/gate: generalize gate for different kinds of subsystems

    Giedrius Statkevičius committed Mar 15, 2019
    Configuration menu
    Copy the full SHA
    70b115d View commit details
    Browse the repository at this point in the history

  5. store/limiter: remove non-nil check

    Giedrius Statkevičius committed Mar 15, 2019
    Configuration menu
    Copy the full SHA
    36f1153 View commit details
    Browse the repository at this point in the history

  6. CHANGELOG: fixes

    Giedrius Statkevičius committed Mar 15, 2019
    Configuration menu
    Copy the full SHA
    9b74bbe View commit details
    Browse the repository at this point in the history

  7. store/limiter: convert failedCounter to non-ptr

    Giedrius Statkevičius committed Mar 15, 2019
    Configuration menu
    Copy the full SHA
    82bdb3c View commit details
    Browse the repository at this point in the history

  8. store/limiter: remove invalid comment

    Giedrius Statkevičius committed Mar 15, 2019
    Configuration menu
    Copy the full SHA
    4d8420f View commit details
    Browse the repository at this point in the history

Commits on Mar 18, 2019

  1. *: update according to review comments

    Giedrius Statkevičius committed Mar 18, 2019
    Configuration menu
    Copy the full SHA
    590b9a6 View commit details
    Browse the repository at this point in the history

  2. CHANGELOG: update

    Giedrius Statkevičius committed Mar 18, 2019
    Configuration menu
    Copy the full SHA
    3f40bac View commit details
    Browse the repository at this point in the history

  3. *: fix according to review

    Giedrius Statkevičius committed Mar 18, 2019
    Configuration menu
    Copy the full SHA
    f4734e5 View commit details
    Browse the repository at this point in the history

  4. *: fix according to review

    Giedrius Statkevičius committed Mar 18, 2019
    Configuration menu
    Copy the full SHA
    d6c1534 View commit details
    Browse the repository at this point in the history

  5. *: make docs

    Giedrius Statkevičius committed Mar 18, 2019
    Configuration menu
    Copy the full SHA
    1147acd View commit details
    Browse the repository at this point in the history

  6. CHANGELOG: clean up

    Giedrius Statkevičius committed Mar 18, 2019
    Configuration menu
    Copy the full SHA
    1d0fad3 View commit details
    Browse the repository at this point in the history

  7. CHANGELOG: update

    @GiedriusS
    GiedriusS committed Mar 18, 2019
    Configuration menu
    Copy the full SHA
    ef4a51e View commit details
    Browse the repository at this point in the history

Commits on Mar 20, 2019

  1. Merge remote-tracking branch 'origin' into feature/store_sample_limit

    Giedrius Statkevičius committed Mar 20, 2019
    Configuration menu
    Copy the full SHA
    48141fd View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2019

  1. *: queries_in_flight_total -> queries_in_flight

    @GiedriusS
    GiedriusS committed Mar 21, 2019
    Configuration menu
    Copy the full SHA
    c9a7d83 View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2019

  1. Merge branch 'master' into smpl_limit

    Giedrius Statkevičius committed Mar 22, 2019
    Configuration menu
    Copy the full SHA
    d71f1d8 View commit details
    Browse the repository at this point in the history

  2. store/bucket: do not wraper samplesLimiter error 

    The original error already informs us about what is going wrong.
    Giedrius Statkevičius committed Mar 22, 2019
    Configuration menu
    Copy the full SHA
    280a8ca View commit details
    Browse the repository at this point in the history

  3. store/bucket: err -> errors.Wrap 

    It's still useful to know that we are talking about samples here
exactly.
    Giedrius Statkevičius committed Mar 22, 2019
    Configuration menu
    Copy the full SHA
    11c4b18 View commit details
    Browse the repository at this point in the history

Commits on Mar 23, 2019

  1. store: make store.grpc.series-max-concurrency 20 by default 

    Setting it to 0 by default doesn't make sense since the Go channel
becomes unbuffered and all queries will timeout. Set it to 20 by default
since that's the limit on Thanos Query and naturally there won't be more
than 20 by default so it's good.
    @GiedriusS
    GiedriusS committed Mar 23, 2019
    Configuration menu
    Copy the full SHA
    31a8346 View commit details
    Browse the repository at this point in the history

  2. CHANGELOG: add warning about new limit

    @GiedriusS
    GiedriusS committed Mar 23, 2019
    Configuration menu
    Copy the full SHA
    6e98dfd View commit details
    Browse the repository at this point in the history