topic Re: How to create a missing data alert based on custom events received in Dynatrace ? in Alerting

How to create a missing data alert based on custom events received in Dynatrace?

gilles_tabary — Thu, 18 Dec 2025 11:02:07 GMT

Hello.

On Managed 1.322.58.

We made an OpenShift cluster send its AlertManager events to Dynatrace through extension "Prometheus Alertmanager (v2)." Fine.

We get events on custom device. For example "Watchdog". Fine.

Entity Alertmanager - openshift-monitoring Time today, 10:25 - 12:25 (2h) dt.event.impact_level Infrastructure dt.event.source Alertmanager dt.event.allow_davis_merge false dt.event.timeout 120 dt.event.description This is an alert meant to ensure that the entire alerting pipeline is functional. This alert is always firing, therefore it should always be firing in Alertmanager and always fire against a receiver. There are integrations with various notification mechanisms that send a notification when this alert is not firing. For example the "DeadMansSnitch" integration in PagerDuty. dt.event.allow_frequent_issue_detection false dt.event.group_label Custom info dt.event.title Watchdog Source Alertmanager

Question : How can we alert in case we don't get the Watchdog event?

Reformulated question : How to create a metric, based on this event, so I can create a metric event and subsequently an alert on missing data ?

I could not find things in the Metric Advanced Search.

Re: How to create a missing data alert based on custom events received in Dynatrace ?

AntonPineiro — Thu, 25 Sep 2025 16:21:02 GMT

Hi,

I have never tried that extension, but looks like you are selected info severity.

Can you try "critical"?

Best regards

Re: How to create a missing data alert based on custom events received in Dynatrace ?

gilles_tabary — Thu, 25 Sep 2025 16:39:00 GMT

Hi.

This parameter you are pointing tells Dynatrace to create a Problem in case an event of severity critical comes in. Is already activated. Works fine. But irrelevent. 😀

Question is : how to detect an *absence* of expected event ? Aka : watchdog alerting in case of missing data.

Re: How to create a missing data alert based on custom events received in Dynatrace ?

AntonPineiro — Thu, 25 Sep 2025 16:59:29 GMT

Hi,

Sorry, I had not catched your point before.

I would say, using Metrics API - POST ingest data points for having data point ingested.

Alert of missing data will create incident without wait or delays, it means, if you do not have a data point every minute, it will create a problem.

You can support this product idea if it would be useful for your use case.

Best regards

Re: How to create a missing data alert based on custom events received in Dynatrace ?

gilles_tabary — Tue, 30 Sep 2025 12:33:57 GMT

Response from support 30/09/2025 : Not possible.

Thank you for reaching out. To my knowledge, we do not have any alerting mechanism when an event does not happen, apart from the “Alert on missing data” switch on metric events configuration. While we do offer methods to create new metrics (to later use with “Alert on missing data” switch to achieve this), such as calculated metric based on an analysis, or creating a metric based on logs , the scenario you described is not applicable to either of these existing capabilities and is therefore not currently supported. I have already reached out to both the Extension and Anomaly Detection teams summarizing the case, and their feedback aligns with this conclusion.