Alerting
Questions about alerting and problem detection in Dynatrace.
cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Improving Dynatrace Alert Speed: From Log Arrival to Notification

AK
Pro

Hi Folks,

have a Davis Anomaly Detection rule configured on SNMP logs, with a workflow in place to forward notifications.

Issue:

  • When a log matches the Davis Anomaly Detection rule, a problem isn’t created for about 4-5 minutes.
  • As a result, the notification reaches the relevant team roughly 5 minutes after the log arrives.

Observed timeline example:

  • Log arrival in Dynatrace: 06:36:31
  • Event analysis start: 06:36
  • Problem created/delivered: 06:39
  • Workflow execution completed: 06:40 (the workflow typically takes 30–50 seconds)

Is there a way to reduce the end-to-end delay to 1–2 minutes?

Regards,
AK

4 REPLIES 4

p_devulapalli
Leader

@AK The delay looks to be during event analysis , are you using the log events or Davis anomaly detection app rule?

Phani Devulapalli

@p_devulapalliI'm pulling logs and creating a time series to trigger alerts—here's the example query I'm using in Davis Anomaly Detection.

fetch logs
| filter matchesValue (log.source, "trap") and matchesValue (trapoid, "Critical")
| makeTimeseries count(), by:{Device, Message, device.address,  dt.source_entity,trapoid},interval:1m

 

@AK Do you have any of the below set to longer duration which can impact the time the problem is generated

p_devulapalli_0-1770602700395.png

 

Phani Devulapalli

AK
Pro

@p_devulapalli Thanks for looking into this. I had been using Davis Anomaly Detection, but after switching to the Davis included in the open pipeline, the delay dropped dramatically to about one to two minutes. Attached is a screenshot of the configuration.

AK_0-1771396732810.png

 Thanks again for your helpful insights and assistance.

Regards,
AK

Featured Posts