Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mimir query engine: support query cancellation and timeout #8197

Merged

Conversation

charleskorn
Copy link
Contributor

@charleskorn charleskorn commented May 28, 2024

What this PR does

This PR adds support for cancelling queries (with Query.Cancel()) and configuring a timeout for queries (with EngineOpts.Timeout) in the Mimir query engine.

Which issue(s) this PR fixes or relates to

(none)

Checklist

  • Tests updated.
  • [n/a] Documentation added.
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
  • [n/a] about-versioning.md updated with experimental features.

@charleskorn charleskorn marked this pull request as ready for review May 28, 2024 12:16
@charleskorn charleskorn requested a review from a team as a code owner May 28, 2024 12:16
Copy link
Collaborator

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! This was easy to review.

pkg/streamingpromql/engine_test.go Show resolved Hide resolved
pkg/streamingpromql/engine_test.go Outdated Show resolved Hide resolved
Copy link
Contributor

@jhesketh jhesketh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, but can we test for errQueryClosed and errQueryFinished?

@charleskorn charleskorn force-pushed the charleskorn/mimir-query-engine-cancellation-and-timeout branch from ab6180d to a41284a Compare May 29, 2024 06:11
@charleskorn
Copy link
Contributor Author

can we test for errQueryClosed and errQueryFinished?

Testing for errQueryClosed is tricky: closing the query while it's executing will lead to unpredictable results, but once the query has finished executing then the context will be cancelled with errQueryFinished.

Testing for errQueryFinished should be straightforward enough, I'll add that now.

@charleskorn charleskorn merged commit de0f9c5 into main May 30, 2024
29 checks passed
@charleskorn charleskorn deleted the charleskorn/mimir-query-engine-cancellation-and-timeout branch May 30, 2024 01:30
narqo pushed a commit to narqo/grafana-mimir that referenced this pull request Jun 6, 2024
)

* Add support for cancelling queries.

* Respect the configured query timeout.

* Update changelog entry

* Use `storage.ErrSeriesSet`

* Ensure we don't leak contexts

* Check for context cancellation in `Selector.Next`

* Add test case to ensure that query execution context is cancelled after evaluation finishes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants