02 Jul 2025 11:48 AM - edited 04 Jul 2025 05:37 PM
Hi!
We've set up a custom metric that contains the name of an external entity (API Gateway) and its health status, like this:
0: unhealthy
1: healthy
Example:
We are receiveing and ingesting multiple same values over time for a given entity, for example:
Entity 1 - healthy (value: 1) -> everything OK, don't send notification
Entity 2 - healthy (value: 1) -> everything OK, don't send notification
Entity 1 - healthy (value: 1) -> everything OK, don't send notification
Entity 1 - unhealthy (value: 0) -> send notification as it is unhealthy
Entity 3 - healthy (value: 1) -> everything OK, don't send notification
Entity 1 - unhealthy (value: 0) -> don't send any notification because we've already did it
Entity 1 - healthy (value: 1) -> send a solved problem notification, as it became healthy again
So, we need to alert only on health status changes:
The trick here is that the email notification should be sent to the team responsible for that entity, which will be different on each case. Alerting Profile + Problem notifications doesn't seem to be able to specify variable destinations to send emails. Workflows have, but then I don't know how to send only emails when health changes and not for every redundant event.
Anyone have some advice in how can we set that up?
***UPDATE: Metric events doesn't seem to provide enough customization for that. Could be Open Pipelines the answer?***
Thanks in advance!
02 Jul 2025 12:39 PM - edited 02 Jul 2025 12:39 PM
02 Jul 2025 02:28 PM
Hi @AntonPineiro,
I've tried Metric Events, but I see no way to have the logic to fit these requirements:
API GW 1 - healthy event (value: 1) -> IGNORE (new event but it is healthy)
API GW 2 - healthy event (value: 1) -> IGNORE (new event but it is healthy)
API GW 1 - healthy event (value: 1) -> IGNORE (repeated event!)
API GW 1 - unhealthy event (value: 0) -> send notification as it is unhealthy
API GW 3 - healthy event (value: 1) -> IGNORE (new event but it is healthy)
API GW 1 - unhealthy event (value: 0) -> IGNORE (repeated event!)
API GW 1 - healthy event (value: 1) -> send a solved problem notification, as it became healthy again
03 Jul 2025 08:00 AM
Hi,
Check sliding window https://docs.dynatrace.com/docs/shortlink/anomaly-detection-configuration#sliding-window
Best regards
03 Jul 2025 04:37 PM
Is not doable. Shortest dealerting sliding window is 3 minute. So the unhealthy API Gateway will take 3 minutes at minimum to close the open problem.
04 Jul 2025 09:29 AM
Hi,
It means, that is how Dynatrace works. If that does not fit your use case, that cannot be done today.
Best regards
02 Jul 2025 02:23 PM
After setting up the metric event, you can link our event to the desired alerting profile either by tagging the metric event accordingly, or by using a custom rule within the alerting profile to directly target the problem name you’ve configured.
02 Jul 2025 02:30 PM
Hi @lubrman thanks, but the problem is exactly on how to set up the metric event or the alerting mechanism (whether is an Anomaly Detector or a Metric Event, or a Workflow, or whatever way).
Next steps to link the Alerting Profile are clear.
03 Jul 2025 06:34 AM
And a similar format doesn’t work for you when searching in the Data Explorer?
You can then specifically define the services you want to monitor — you can use the same selector in the metric event as well.
custome.api.gateway.healt.status:filter(and(or(in("api_name",entitySelector("type(api_name),entityName.equals(~"Entity 1~")"))))):splitBy("dt.entity.process_group_instance"):sort(value(auto,descending))