Alerting
Questions about alerting and problem detection in Dynatrace.
cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Davis Anomaly Detector - “used/available (GiB) Disk spae” in the problem without creating new problems

Goal

I want a Davis anomaly detector for disk usage % (e.g., alert when dt.host.disk.used.percent is high), and I’d like the problem to also show used and available capacity (in GiB) for the affected disk.

 

The issue

When I attach “used/avail” as scalar context via a join in the detector DQL, Dynatrace starts opening new problems whenever those values change (e.g., ±1 GiB) — even though it’s the same host/disk and the same underlying breach.

 

Here’s the DQL I ended up with (simplified):


timeseries {
value = avg(dt.host.disk.used.percent)
}, by:{ `dt.entity.host_group`, `dt.entity.host`, `dt.entity.disk`, dt.cost.costcenter }

| fieldsAdd
host_group_name = entityName(`dt.entity.host_group`),
host_name = entityName(`dt.entity.host`),
disk_name = entityName(`dt.entity.disk`)
| filter disk_name != "/var/log/audit"


| join [
timeseries {
used_ts = avg(dt.host.disk.used),
avail_ts = avg(dt.host.disk.avail)
}, by:{ `dt.entity.disk` }
| fields
`dt.entity.disk`,
used_last_bytes = arrayLast(arrayRemoveNulls(used_ts)),
avail_last_bytes = arrayLast(arrayRemoveNulls(avail_ts))
],
on:{ `dt.entity.disk` }, kind: leftOuter


| fieldsAdd
used_last_GiB = right.used_last_bytes / 1073741824.0,
avail_last_GiB = right.avail_last_bytes / 1073741824.0

| fieldsAdd used_GiB = round(used_last_GiB, decimals:0),
avail_GiB = round(avail_last_GiB, decimals:0)
| fieldsAdd total_GiB = used_GiB + avail_GiB

| fieldsRemove right.used_last_bytes, right.avail_last_bytes, used_last_GiB, avail_last_GiB

 

Observed behavior

Every time used_GB / avail_GB changes, the detector starts a new problem (instead of keeping one open), which creates noise.

Is there a supported/best-practice way to include dynamic context (used/avail) in the problem without causing Davis to treat it like a “new series” and open a new problem?

 

2 REPLIES 2

O11y_Noob
Participant

are you using {dims:total_GiB} in your problem ? then what i understood is anything in {dims:xxx} changes when used under the problem , DT considers it as change and closes and opens a new prob from this thread that i am facing , try with {fields: as suggested , but it dint help in my case} 

alerting site wise - Dynatrace Community

p_devulapalli
Leader

@patryk_ozimek2 I've come across this limitation before and this is a expected behavior if you include dimensions that change . Here is an idea that I raised for the same

https://community.dynatrace.com/t5/Product-ideas/Include-variable-dimensions-in-event-description-fo...

 

Phani Devulapalli

Featured Posts