Fixed compact failed due to duplicate labels #542
Conversation
Signed-off-by: jojohappy <sarahdj0917@gmail.com>
jojohappy
force-pushed
the
fix_compactor_err_with_duplicate_labels
branch
from
67b402f
to
4d8f023
Compare
There was a problem hiding this comment.
Not sure about all of this, sure it unblock peope temporarily until the problem will not grow to enormous sizes (label will be duplicated 1000 times, because of 1000 compactions the does not handle this gracefully).
The questions is again.. why this happens.. who creates that broken blocks. (: I think we need to rethink the whole block sanity problem and how we handle this.
| @@ -282,7 +283,9 @@ func GatherIndexIssueStats(logger log.Logger, fn string, minTime int64, maxTime | |||
| } | |||
| l0 := lset[0] | |||
| for _, l := range lset[1:] { | |||
| if l.Name <= l0.Name { | |||
| if l.Name == l0.Name { | |||
There was a problem hiding this comment.
It only makes sense if l.Value == l0.Value as well I guess.
There was a problem hiding this comment.
But we would hit the case that with same label name and different value. I think it has been not make sense with duplicate label name.
|
Totally agree with you @bwplotka. So should we open a ticket in upstream (Prometheus)? |
|
Yes, if you can repro the duplicated label problem. Let's close this for now until we have a common way of handling those issues. |
Signed-off-by: jojohappy sarahdj0917@gmail.com
Changes
fixes #497
Just print out an error log to prompt the series have duplicate labels. I think it is an unusual case, so I do nothing for the that.
Verification