DQL - Anomaly detector interval 1m

[AntonPineiro]

Hi,

I would like to create an anomaly detector disk alert when disk is higher than some value. I am trying it using this DQL:

timeseries t_disk = avg(dt.host.disk.used.percent),
filter: {
  matchesValue(role, "ORACLE") AND
  matchesValue(entityAttr(dt.entity.disk, "entity.name"), "/opt")
}, by:{dt.entity.host, dt.entity.disk}, interval: 1m
| fieldsAdd server = entityName(dt.entity.host)
| fieldsAdd disk = entityName(dt.entity.disk)
| fieldsAdd `disk_used` = arrayAvg(t_disk)
| fieldsRemove interval, t_disk
| join [
    timeseries disk.used_sum = avg(dt.host.disk.used), disk.avail_sum = avg(dt.host.disk.avail), by:{dt.entity.disk}
    | fieldsAdd total_disk = toLong((arrayAvg(disk.used_sum) + arrayAvg(disk.avail_sum)))
    | fieldsKeep total_disk, dt.entity.disk
  ], on:{dt.entity.disk}
| fieldsAdd `disk_size`  = right.total_disk
| fieldsRemove right.dt.entity.disk, right.total_disk
| filter `disk_size` > 5000000000

I am getting some interval error:

But I think issue is another thing. Do you have some suggestion? Have you received that error message before?

Best regards

[t_pawlak · Accepted solution]

Hi,
I thinkk the problem is here:
arrayAvg(t_disk) collapses the time series into a single value and fieldsRemove interval removes the interval information required by the detector

So for anomaly detection, your query should keep the original time series in the final output, and only use the join for filtering/enrichment. Try this:

timeseries t_disk = avg(dt.host.disk.used.percent),
filter: {
  matchesValue(entityAttr(dt.entity.disk, "entity.name"), "/boot")
},
by:{dt.entity.host, dt.entity.disk},
interval: 1m
| join [
    timeseries disk.used_sum = avg(dt.host.disk.used),
              disk.avail_sum = avg(dt.host.disk.avail),
              by:{dt.entity.disk},
              interval: 1m
    | fieldsAdd total_disk = toLong(arrayAvg(disk.used_sum) + arrayAvg(disk.avail_sum))
    | fieldsKeep total_disk, dt.entity.disk
  ], on:{dt.entity.disk}
| fieldsAdd server = entityName(dt.entity.host)
| fieldsAdd disk = entityName(dt.entity.disk)
| fieldsAdd disk_size = right.total_disk

Of course change filters 😉 

[AntonPineiro]

Hi,

You are right, before query should work if this line is removed:

| fieldsRemove interval, t_disk

 Thank you!

Best regards