Hi,
I have created anomaly detection to capture specific error from the logs.In that I have defined dql that get timeseries data for the error occurrence for particular k8s application. Statis threshold is defined.
In the anomaly list , status shows “success”. I verified target error has occurred , seen in the logs as well.But somehow I am not able to see my custom event in in grail-> events.
Can somebody please help me to find all the custom events in grail which are related to all the Anomaly detection.
Note - My final goal is to trigger workflow if custom events occurs.
Hi,
dt.system.events will usually not show the actual custom event raised by anomaly detection. That bucket is mainly for detector execution/status information, such as ANOMALY_DETECTOR_STATUS_EVENT and ANALYZER_EXECUTION_EVENT—for example, whether the detector ran successfully, timed out, or produced warnings
If the anomaly detection configuration really raised an alert, I’d look for it in Davis events, for example:
and then narrow it down using your event template fields / event name if available. Dynatrace treats events from custom alerts as Davis events, which can then be correlated into Problems
So if your final goal is to trigger a workflow, I’d first verify whether the anomaly detector is producing a `CUSTOM_ALERT` Davis event, and then decide whether to trigger on the event itself or on the correlated problem.
I referred System event models. , checked below DQL, and found my task is failed with reason “The task was disabled by the service operator to maintain system stability. Contact support if the task keeps failing.”
“The task was disabled by the service operator to maintain system stability. Contact support if the task keeps failing.”
it looks like the specific anomaly detection task was automatically disabled by Dynatrace safeguards, most likely because the detector execution was considered too expensive or unstable.
So I would suggest review the anomaly detector query and try to reduce its cost, like narrow the timeframe, or use some filters. Make it less expensive.
And if the issue continues, opening a Dynatrace Support case, because the message suggests a service-side safeguard.
hi! in addition to all the useful comments, I wanted to share this new page in the documentation that helps better understand how everything fit together: Alerting and notifications — Dynatrace Docs including this graphic.