Support UTF-8 label matchers: Use compat package in Alertmanager server #3567

grobinson-grafana · 2023-10-20T16:03:21Z

This pull request adds use of the compat package in Alertmanager server that will allow users to switch between the new matchers/parse parser and the old pkg/labels parser. The new matchers/parse parser uses a fallback mechanism where if the input cannot be parsed in the new parser it then attempts to use the old parser. If an input is parsed in the old parser but not the new parser then a warning log is emitted.

api/v2/api.go

grobinson-grafana · 2023-11-14T17:22:15Z

silence/silence.go

+}
+
+// InitFromFlags initializes the validation function from the flagger.
+func InitFromFlags(l log.Logger, f featurecontrol.Flagger) {


I found an interesting issue here! If you create a silence with UTF-8 matchers and then restart Alertmanager with the "classic-matchers-parsing" feature flag, you can no longer edit or expire the silence because the validation function returns an error. However, the silence will still expire once it's expiration time has elapsed.

I've attempted to fix that with this commit here.

Great Find! I'll take a look once we have the other changes in.

grobinson-grafana · 2023-11-16T16:03:02Z

@gotjosh let me know when you have some time and we can review this together?

gotjosh

Great job so far, please take a look at my comments.

api/v2/api.go

gotjosh · 2023-11-21T16:11:24Z

cli/alert_query.go

@@ -82,7 +83,7 @@ func (a *alertQueryCmd) queryAlerts(ctx context.Context, _ *kingpin.ParseContext
 		m := a.matcherGroups[0]
 		_, err := compat.Matcher(m)
 		if err != nil {
-			a.matcherGroups[0] = fmt.Sprintf("alertname=%s", m)
+			a.matcherGroups[0] = fmt.Sprintf("alertname=%s", strconv.Quote(m))


This is not related - just something we missed from before, right? We had before silence_add, silence_query and alert_add so I guess it makes sense.

Yes, correct! I noticed it as the changes I made in this PR caused a test to fail.

gotjosh · 2023-11-21T16:35:33Z

test/with_api_v2/acceptance/silence_test.go

+
+	_, err := am.Client().Silence.PostSilences(silenceParams)
+	require.NoError(t, err)
+}


Can we add a test for the case that you found - where you create an utf-8 silence, restart the server with the classic flag and try to operate on it?

I'll take a look. I might need to make some larger changes to acceptance.go to support restarting clusters, as it seems at present stopping a cluster deletes all it's data.

test/with_api_v2/acceptance/api_test.go

gotjosh · 2023-11-21T16:52:53Z

test/with_api_v2/acceptance/silence_test.go

+	at := NewAcceptanceTest(t, &AcceptanceOpts{
+		Tolerance: 150 * time.Millisecond,
+	})


I'm confused - and running this test in isolation seems to agree with me. Where exactly do we make this test accept Unicode characters?

Remember the default mode of Alertmanager (unless you use classic-mode) is UTF-8, with fallback to classic mode. Makes me think I should rename utf8-mode to utf8-strict-mode.

Thanks! I completely forgot about this haha.

types/types.go

gotjosh · 2023-11-21T17:13:45Z

silence/silence.go

+}
+
+// InitFromFlags initializes the validation function from the flagger.
+func InitFromFlags(l log.Logger, f featurecontrol.Flagger) {


Great Find! I'll take a look once we have the other changes in.

silence/silence.go

cmd/alertmanager/main.go

gotjosh

Great job! I only have 2 files left to review - silence and silce_test.

api/api.go

api/v2/api.go

gotjosh · 2023-11-22T15:59:35Z

test/with_api_v2/acceptance/utf8_test.go

+	// Can get same alert from the API.
+	resp, err := am.Client().Alert.GetAlerts(nil)
+	require.NoError(t, err)
+	require.Len(t, resp.Payload, 1)
+	require.Equal(t, labels, resp.Payload[0].Labels)
+
+	// Can filter alerts on UTF-8 labels.
+	getAlertParams := alert.NewGetAlertsParams()
+	getAlertParams = getAlertParams.WithFilter([]string{"00=b", "Σ=c", "\"\\xf0\\x9f\\x99\\x82\"=dΘ"})
+	resp, err = am.Client().Alert.GetAlerts(getAlertParams)
+	require.NoError(t, err)
+	require.Len(t, resp.Payload, 1)
+	require.Equal(t, labels, resp.Payload[0].Labels)
+
+	// Can get same alert in alert group from the API.
+	alertGroupResp, err := am.Client().Alertgroup.GetAlertGroups(nil)
+	require.NoError(t, err)
+	require.Len(t, alertGroupResp.Payload, 1)
+	require.Len(t, alertGroupResp.Payload[0].Alerts, 1)
+	require.Equal(t, labels, alertGroupResp.Payload[0].Alerts[0].Labels)
+
+	// Can filter alertGroups on UTF-8 labels.
+	getAlertGroupsParams := alertgroup.NewGetAlertGroupsParams()
+	getAlertGroupsParams.Filter = []string{"00=b", "Σ=c", "\"\\xf0\\x9f\\x99\\x82\"=dΘ"}
+	alertGroupResp, err = am.Client().Alertgroup.GetAlertGroups(getAlertGroupsParams)
+	require.NoError(t, err)
+	require.Len(t, alertGroupResp.Payload, 1)
+	require.Len(t, alertGroupResp.Payload[0].Alerts, 1)
+	require.Equal(t, labels, alertGroupResp.Payload[0].Alerts[0].Labels)


Great job! ❤️

gotjosh

LGTM but please take a look at my comment wrt the silences validation.

I think we're missing three things at this point:

Make the CI pass
The test to catch the removal of validation in expire
The avoidance of copy paste code.

silence/silence.go

gotjosh · 2023-11-23T09:55:27Z

silence/silence.go

+func validateUTF8Matcher(m *pb.Matcher) error {
+	if !utf8.ValidString(m.Name) {
+		return fmt.Errorf("invalid label name %q", m.Name)
+	}
+	switch m.Type {
+	case pb.Matcher_EQUAL, pb.Matcher_NOT_EQUAL:
+		if !utf8.ValidString(m.Pattern) {
+			return fmt.Errorf("invalid label value %q", m.Pattern)
+		}
+	case pb.Matcher_REGEXP, pb.Matcher_NOT_REGEXP:
+		if !utf8.ValidString(m.Pattern) {
+			return fmt.Errorf("invalid regular expression %q", m.Pattern)
+		}
+		if _, err := regexp.Compile(m.Pattern); err != nil {
+			return fmt.Errorf("invalid regular expression %q: %s", m.Pattern, err)
+		}
+	default:
+		return fmt.Errorf("unknown matcher type %q", m.Type)
+	}
+	return nil
+}
+


I have my reservations when it comes to this code - to me, this feels wrong. We're accepting matches as user input from the silences API in a structured format:

"matchers": [ { "name": "string", "value": "string", "isRegex": true, "isEqual": true } ],

In my opinion, this validation should happen at an API level and ideally make the input matchers go through the parser, by taking each parser and constructing a string that would you give you the struct directly.

My main worry is that right now validation for entities is spreadout in multiple layers which makes it difficult to track who validates what.

Not something that we need to solve right now, but it's worth thinking about.

gotjosh

LGTM - please take a look at my comments but overall I have no further blocking questions.

Great work and thank you very much for your contribution.

silence/silence.go

gotjosh · 2023-11-24T09:09:31Z

I've merged #2469, can you please rebase so that I can also merge this one?

Signed-off-by: George Robinson <george.robinson@grafana.com>

This commit adds isDelete to setSilence to skip validation when expiring silences. This prevents an issue where a silence that is no longer valid (i.e. the rules have changed in a newer version of Alertmanager) cannot be expired via the API or UI. Signed-off-by: George Robinson <george.robinson@grafana.com>

Signed-off-by: George Robinson <george.robinson@grafana.com>

grobinson-grafana · 2023-11-24T09:49:21Z

I've merged #2469, can you please rebase so that I can also merge this one?

Done! 🙂

gotjosh

LGTM

grobinson-grafana mentioned this pull request Oct 20, 2023

Epic: Support UTF-8 characters in Alertmanager label matchers #3486

Closed

4 tasks

grobinson-grafana changed the title ~~Use compat package in Alertmanager server~~ Support UTF-8 label matchers: Use compat package in Alertmanager server Oct 20, 2023

grobinson-grafana marked this pull request as draft October 20, 2023 16:04

grobinson-grafana force-pushed the grobinson/use-adapter-alertmanager-server branch from cd3904e to 67c0042 Compare November 1, 2023 10:28

grobinson-grafana commented Nov 1, 2023

View reviewed changes

api/v2/api.go Outdated Show resolved Hide resolved

grobinson-grafana marked this pull request as ready for review November 2, 2023 17:13

grobinson-grafana marked this pull request as draft November 3, 2023 20:02

grobinson-grafana mentioned this pull request Nov 6, 2023

Update Alertmanager vendor grafana/mimir#6560

Merged

3 tasks

grobinson-grafana force-pushed the grobinson/use-adapter-alertmanager-server branch 3 times, most recently from df1e680 to 3f30b33 Compare November 14, 2023 17:16

grobinson-grafana commented Nov 14, 2023

View reviewed changes

grobinson-grafana force-pushed the grobinson/use-adapter-alertmanager-server branch from 9ec6b89 to 7261a7f Compare November 15, 2023 13:14

grobinson-grafana marked this pull request as ready for review November 15, 2023 13:17

grobinson-grafana force-pushed the grobinson/use-adapter-alertmanager-server branch from 64ac0a0 to bfb4b78 Compare November 21, 2023 11:27

gotjosh reviewed Nov 21, 2023

View reviewed changes

gotjosh mentioned this pull request Nov 21, 2023

Deprecate and remove api/v1/ #2970

Merged

3 tasks

grobinson-grafana force-pushed the grobinson/use-adapter-alertmanager-server branch 2 times, most recently from 2c60bec to ee2e8da Compare November 22, 2023 14:42

grobinson-grafana requested a review from gotjosh November 22, 2023 15:12

gotjosh reviewed Nov 22, 2023

View reviewed changes

gotjosh reviewed Nov 23, 2023

View reviewed changes

grobinson-grafana force-pushed the grobinson/use-adapter-alertmanager-server branch from 423ddc7 to 422e80f Compare November 23, 2023 10:34

grobinson-grafana requested a review from gotjosh November 23, 2023 16:08

grobinson-grafana force-pushed the grobinson/use-adapter-alertmanager-server branch from f2529d0 to a33c000 Compare November 23, 2023 16:35

gotjosh approved these changes Nov 23, 2023

View reviewed changes

silence/silence.go Show resolved Hide resolved

grobinson-grafana added 2 commits November 24, 2023 09:21

Use compat package in Alertmanager server

3f96a45

Signed-off-by: George Robinson <george.robinson@grafana.com>

Fix tests

48368ab

Signed-off-by: George Robinson <george.robinson@grafana.com>

grobinson-grafana added 24 commits November 24, 2023 09:21

Add acceptance tests for creating UTF-8 alerts

b9965d1

Signed-off-by: George Robinson <george.robinson@grafana.com>

Add tests for classic alerts

312a9da

Signed-off-by: George Robinson <george.robinson@grafana.com>

Add acceptance tests for creating silences

8022958

Signed-off-by: George Robinson <george.robinson@grafana.com>

Fix lint

976b713

Signed-off-by: George Robinson <george.robinson@grafana.com>

Fix use of renamed feature flags

fa2bca6

Signed-off-by: George Robinson <george.robinson@grafana.com>

Inject feature flags into APIs and Silencer

a38aeb4

Signed-off-by: George Robinson <george.robinson@grafana.com>

Fix tests

fc1a2db

Signed-off-by: George Robinson <george.robinson@grafana.com>

Use parseFilters

6916407

Signed-off-by: George Robinson <george.robinson@grafana.com>

Rename UTF-8 mode to UTF-8 strict mode

7c5b767

Signed-off-by: George Robinson <george.robinson@grafana.com>

Move UTF-8 tests to separate file

38a5bcd

Signed-off-by: George Robinson <george.robinson@grafana.com>

Add tests for filtering alerts and silences

cda2a30

Signed-off-by: George Robinson <george.robinson@grafana.com>

Fix license

fb98219

Signed-off-by: George Robinson <george.robinson@grafana.com>

Fix lint

5ab3d73

Signed-off-by: George Robinson <george.robinson@grafana.com>

Add test for filtering alert groups

9fa29ad

Signed-off-by: George Robinson <george.robinson@grafana.com>

Fix lint again

9da9da6

Signed-off-by: George Robinson <george.robinson@grafana.com>

Fix tests

e3c7ba8

Signed-off-by: George Robinson <george.robinson@grafana.com>

Fix comments

51187ea

Signed-off-by: George Robinson <george.robinson@grafana.com>

Feedback

8c86375

Signed-off-by: George Robinson <george.robinson@grafana.com>

Fix

d5aebdf

Signed-off-by: George Robinson <george.robinson@grafana.com>

Rename to skipValidate

969ccc3

Signed-off-by: George Robinson <george.robinson@grafana.com>

Feedback

7fa308e

Signed-off-by: George Robinson <george.robinson@grafana.com>

Fix rebase mistake

c3655b4

Signed-off-by: George Robinson <george.robinson@grafana.com>

Fix deletion of v1

318b822

Signed-off-by: George Robinson <george.robinson@grafana.com>

grobinson-grafana force-pushed the grobinson/use-adapter-alertmanager-server branch from 62b5ade to 318b822 Compare November 24, 2023 09:29

gotjosh approved these changes Nov 24, 2023

View reviewed changes

gotjosh merged commit 70bd5da into prometheus:main Nov 24, 2023
11 checks passed

alanprot mentioned this pull request Apr 2, 2024

Chore: Update Prometheus cortexproject/cortex#5841

Merged

grobinson-grafana deleted the grobinson/use-adapter-alertmanager-server branch April 16, 2024 14:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support UTF-8 label matchers: Use compat package in Alertmanager server #3567

Support UTF-8 label matchers: Use compat package in Alertmanager server #3567

grobinson-grafana commented Oct 20, 2023 •

edited

Loading

grobinson-grafana Nov 14, 2023

grobinson-grafana Nov 15, 2023

gotjosh Nov 21, 2023

grobinson-grafana commented Nov 16, 2023

gotjosh left a comment

gotjosh Nov 21, 2023

grobinson-grafana Nov 21, 2023

gotjosh Nov 21, 2023

grobinson-grafana Nov 22, 2023

gotjosh Nov 21, 2023

grobinson-grafana Nov 22, 2023

gotjosh Nov 22, 2023

gotjosh Nov 21, 2023

gotjosh left a comment

gotjosh Nov 22, 2023

gotjosh left a comment

gotjosh Nov 23, 2023

gotjosh left a comment

gotjosh commented Nov 24, 2023

grobinson-grafana commented Nov 24, 2023

gotjosh left a comment

Support UTF-8 label matchers: Use compat package in Alertmanager server #3567

Support UTF-8 label matchers: Use compat package in Alertmanager server #3567

Conversation

grobinson-grafana commented Oct 20, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

grobinson-grafana commented Nov 16, 2023

gotjosh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gotjosh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gotjosh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gotjosh left a comment

Choose a reason for hiding this comment

gotjosh commented Nov 24, 2023

grobinson-grafana commented Nov 24, 2023

gotjosh left a comment

Choose a reason for hiding this comment

grobinson-grafana commented Oct 20, 2023 •

edited

Loading