query-frontend worker: cancel is not an error

Could be the user closed the tab, or it's a sharded query and some other
shard errored. Whatever the cause, cancellation is not an error.

I expect two effects: the worker won't log "context canceled", and it
will process the next request immediately.

CHANGELOG.md

@@ -79,6 +79,7 @@
 * [ENHANCEMENT] Query-frontend: added experimental `-query-frontend.query-sharding-max-regexp-size-bytes` limit to query-frontend. When set to a value greater than 0, query-frontend disabled query sharding for any query with a regexp matcher longer than the configured limit. #4632
 * [ENHANCEMENT] Query-frontend: improve readability of distributed tracing spans. #4656
 * [ENHANCEMENT] Update Docker base images from `alpine:3.17.2` to `alpine:3.17.3`. #4685
+* [ENHANCEMENT] Query-frontend: don't treat cancel as an error. #4648
 * [BUGFIX] Querier: Streaming remote read will now continue to return multiple chunks per frame after the first frame. #4423
 * [BUGFIX] Store-gateway: the values for `stage="processed"` for the metrics `cortex_bucket_store_series_data_touched` and  `cortex_bucket_store_series_data_size_touched_bytes` when using fine-grained chunks caching is now reporting the correct values of chunks held in memory. #4449
 * [BUGFIX] Compactor: fixed reporting a compaction error when compactor is correctly shut down while populating blocks. #4580

pkg/querier/worker/frontend_processor.go

@@ -14,6 +14,7 @@ import (
 	"github.com/go-kit/log"
 	"github.com/go-kit/log/level"
 	"github.com/grafana/dskit/backoff"
+	commongrpc "github.com/weaveworks/common/grpc"
 	"github.com/weaveworks/common/httpgrpc"
 	"go.uber.org/atomic"
 	"google.golang.org/grpc"
@@ -84,9 +85,10 @@ func (fp *frontendProcessor) processQueriesOnSingleStream(workerCtx context.Cont
 		}
 
 		if err := fp.process(c, inflightQuery); err != nil {
-			level.Error(fp.log).Log("msg", "error processing requests", "address", address, "err", err)
-			backoff.Wait()
-			continue
+			if !commongrpc.IsCanceled(err) {
+				level.Error(fp.log).Log("msg", "error processing requests", "address", address, "err", err)
+				backoff.Wait()
+			}
 		}
 
 		backoff.Reset()