Metric events that sum values in a certain time period?

[AntonioSousa]

I have an extension that sums unavailability timings, in 5 minutes intervals. So, at every 5 minutes I have datapoints like failed 10 seconds in the last 5 minutes. Values are from 0 to 300 seconds, for each 5 minute datapoints

Now, I want to setup an alert, through metric events, if it fails like more than 12 minutes in the last 15 minutes. I have tried, but I have not got there. I have several approaches that get near, but would like to know if someone can nail this one?

[Julius_Loman · Accepted solution]

@AntonioSousa What about using rollup for that?

:rollup(sum,"15m")

This should give you sum of your datapoints from last 15 minutes, and then compare it against fixed value of 12?

[AntonioSousa]

@Julius_Loman,

Spot on! Still testing, but seems it will meet the requirements. Had completely forgotten about all the other transformations that you don't have in the Data Explorer interface!

For all eventually seeing this in the future, if you get the cryptic message
"Illegal transform operator: Usage of RollupOperator{rollup=sum, rate=15m} requires a previous value extraction."

you need to provide an aggregation (eg. avg) which I wasn't using.

One thing I noticed is that in the Alert preview I went from
The number of metric dimension values in the last 24 hours is 748
to
The number of metric dimension values in the last 24 hours (52360) exceeds the limit of 1000. Please refine the filter criteria or use a metric key based query definition.

Seems like only associated with the Alert preview, hope it doesn't impact DDU consumption. I do have real sparse data here (if I don't have failures in each 5 minutes, there is no data point), so not really impacted, besides not getting the Alert preview.

[Julius_Loman]

Sure, you need to extract a value to perform the rollup. About dimensions - you already have that data in Dynatrace, so DDUs are already consumed as they are billed on metric ingestion. Maybe you need to remove dimensions completely or just level some by using :splitBy(...) ?

[AntonioSousa]

@Julius_Loman 

Yes, I do use splitBy(), but for the dimension being splitted, there can be thousands of values. I'm not sure if I'm going to hit some type of limit, and therefore not have certain events being generated. I'm checking with the data, to be sure on that...

[AntonioSousa]

@Julius_Loman,

Just to complement I was affected by throttling. I wrote a little bit about it here:
https://community.dynatrace.com/t5/Dynatrace-tips/Metric-Events-limits/m-p/217854

Do you have an idea why the rollup() affectedthe metric dimension values so much, from 748 to 52360?

[Julius_Loman]

@AntonioSousa can you share your metric expression and also some screenhosts/data?

[TorstenHellwig]

Hmm, now this stops where it's getting interesting. I need to use rollup to smoothen the metric and it let's dimensions explode. I just don't get why.

The expression looks like this: <metric_key>:sum:splitBy("dt_name"):rollup(min,30m).

The dimension has a handful of values only. As soon as I remove the rollup, no problem.

Any idea how to prevent this? I tried reordering the expression, nothing seems to work.