[Alerts Plugin] Resolved action messages are buggy when editing; context variables don't work

[Zacqary]

When I create an alert and change the default message from "Resolved" to something else, the change doesn't work. My alerts will only ever send "Resolved."

When I edit the alert from the Alerts Management screen and try to change the message again, it does change what the server logs out, but:

• Context variables don't work
• Editing the alert again just shows the default Resolved message in the box

Only tested this with the Server Log action type; unsure if it affects others.

[elasticmachine]

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

[pmuellr]

When I create an alert and change the default message from "Resolved" to something else, the change doesn't work. My alerts will only ever send "Resolved."

Did you happen to check the alert to see if the change got set in the action params? Perhaps this is just a UI issue?

When I edit the alert from the Alerts Management screen and try to change the message again, it does change what the server logs out, but:

How did you do the first edit? In-app? I guess I thought your first edit ^^^ was also in alerts management ...

Context variables don't work

This came up when we did the work - the problem is, we don't have any context in this case, we're basically determining that "last time this alert scheduled actions for this instance, this time it didn't". The only context variables we could provide would be the values from the previous run. Which seems either wrong or confusing or both :-). And we are not currently storing these anywhere, making it even harder :-)

There are still variables available, so in theory you could construct a message using the alert params, eg; just not the context variables:

kibana/x-pack/plugins/alerts/server/task_runner/transform_action_params.ts

Lines 45 to 54 in 3651748

	const variables = {
	alertId,
	alertName,
	spaceId,
	tags,
	alertInstanceId,
	context,
	state,
	params: alertParams,
	};

Worth opening an issue on this, I'm sure it will come up again, maybe there's an angle on this that I'm missing ... actually, I'm realizing that we have a context variable in the index threshold alert (currently named function but will be renamed to conditions), intended to be a human readable form of the "conditions" under which the alert is triggered. Built from the params. Super unfortunate that that one is not available, for use in "resolved" actions, since I'd certainly want to use that in the resolved message. 🤔

Editing the alert again just shows the default Resolved message in the box

So that sounds like a UI issue ... and just verified that I'm seeing the same.

Only tested this with the Server Log action type; unsure if it affects others.

No action type will get context variables in their "resolved" actions (but let's explore with a new issue), but I could imagine other actions having similar UI issues (assuming that's a UI issue, and not a server-side issue).

[Zacqary]

Did you happen to check the alert to see if the change got set in the action params? Perhaps this is just a UI issue?

Looks like it was indeed set in the action params in the request sent to the POST /alerts API, when creating a new alert. It's also set in the action params when sending the PUT request to edit the existing alert.

There are still variables available, so in theory you could construct a message using the alert params

Might be able to do that, though for Metrics alerts we do have the verbose context.reason value which is supposed to format the thresholds/current values of an unlimited number of alert conditions in a readable way. You're supposed to be able to see a message like:

Test Alert is resolved

Reason:
system.cpu.user.pct is now below a threshold of 70% (current value is 52.1%);

(and we can't get the current value from the alert params)

I'm actually noticing in my tests that context variables will sometimes work on Resolved actions, and then sometimes they won't. Not sure why.

[pmuellr]

That "reason" looks pretty close to what we have in index threshold context variables as "function" now, but will be "conditions" soon-ish (before 7.11 FF hopefully).

I think that now that there's two of these things, it warrants considering to make it a top-level variable that the alert executor would provide somehow. If we continue with "no context variables are available during resolve", then that's why it would be "top-level" (peer of context); and we'd need to find a way to have the alert executor set it, and we'd likely need to plan on more added later. @mikecote?

[pmuellr]

Actually, now that I think about it, this doesn't need to be in the alert executor, because the idea is that it's a human-readable representation of the alert params, so it could be some thing on the alert itself, not associated with the executor. Could be useful to have this displayed somewhere in the UI as the alert is being edited, or expando section in alert/instances lists or something.

[mikecote]

Looks like there's two separate issues here:

• The message doesn't seem to update when using the resolved action group
• Unable to access context variables

To solve the context variables, we may need to expand defaultActionMessage to work by action group or something. It won't be possible to get the context but maybe building a default action message for the resolved group would work?

In the example above, you could drop the second part and built the rest from the params (ex: system.cpu.user.pct is now below a threshold of 70%).

@pmuellr should we break down #66095 or create a separate issue if we go the message by group path?

[pmuellr]

There are definitely multiple issues here:

• UI issue that Zacqary identified, where re-editing does not show the current value
• the fact that we don't have a great story for defaultActionMessage for resolved; presumably this would be handled by [Alerting][Discuss] allow alerts to create default action parameters #66095
• would it be useful to have a top-level conditions variable provided by alerts based just on the alert params - I just opened [alerts][discuss] should alert types have a top-level action param variable for conditions #84148 for this

I think we should scope this issue to the first one - the UI issue, since we have to, and know how to fix that, and don't yet know what the answers to the others are. I don't