Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

querier: Remove exact chunk duplicates on StoreAPI level, potentially merging frames together. #2546

Closed
bwplotka opened this issue Apr 30, 2020 · 1 comment

Comments

@bwplotka
Copy link
Member

bwplotka commented Apr 30, 2020

This needs some benchmarks as well. Found in #2528

Duplicates are handled ok in terms of data correctness but it's not very efficient. In @SuperQ repro data we found 3x chunk duplications. This is when you have duplicated StoreAPI as well as sidecar + store GW overlap. (: We can totally trim this on proxy level instead of concating all.

AC:

  • proxy.go returns chunks without duplications
  • frames are also "compacted" on up to 1MB (or defined size).
@bwplotka
Copy link
Member Author

proxy.go returns chunks without duplications was done: #2603

The defined size AC was not done and it was actually part of another ticket as well plus the idea was abandoned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant