Alerting for DEM Consumption

[jeeeyps]

Hi team.

I have a requirement to trigger an alert if the monthly DEM consumption reaches 80% of the monthly max of 800k.
I used the metric key dem.units.monthly.consumed and set the threshold too 640000 but it does not trigger the alert.
I tried testing with multiple values and still cannot get it to work.

Can you get further suggestions how to improve my implementation?

[Fin_Ubels]

Hey @jeeeyps 

I've just checked a couple of environments I'm in and don't see a metric called "dem.units.monthly.consumed" in either classic or Grail metrics, is this a custom metric that has been set up within your tenant? Also, when you graph the data for that metric, what does it look like, how far apart are the data points? Is it only recording a value to the metric once a month, once a day, every minute?

[jeeeyps]

Hi @Fin_Ubels 
The metric dem.units.monthly.consumed only throws out a value every 0000 UTC. You can check the screenshot below

Nevertheless, I tried pursuing a different approach and make use of metric selector instead using below:

(builtin:billing.synthetic.actions:splitBy():default(0,always))
+(0.1*(builtin:billing.synthetic.requests:splitBy():default(0,always)))
+(0.1*(builtin:billing.synthetic.external:splitBy():default(0,always)))
+(builtin:billing.apps.web.sessionsWithReplayByApplication:filter(eq("Type","Billed")):splitBy():default(0,always))
+0.25*(builtin:billing.apps.web.sessionsWithoutReplayByApplication:filter(eq("Type","Billed")):splitBy():default(0,always))
+(0.01*(builtin:billing.apps.web.userActionPropertiesByApplication:splitBy():default(0,always)))
+(builtin:billing.apps.mobile.sessionsWithReplayByApplication:filter(eq("Type","Billed")):splitBy():default(0,always))
+0.25*(builtin:billing.apps.mobile.sessionsWithoutReplayByApplication:filter(eq("Type","Billed")):splitBy():default(0,always))
+(0.01*(builtin:billing.apps.mobile.userActionPropertiesByMobileApplication:splitBy():default(0,always)))
+0.25*(builtin:billing.apps.custom.sessionsWithoutReplayByApplication:filter(eq("Type","Billed")):default(0,always))

however, I can't seem to figure out how to get the value for the entire month using this approach so that the alert will trigger based on the monthly consumption. Any advise?

[Yosi_Neuman]

Hi @jeeeyps 

Can you please share the information you see in data explorer about dem.units.monthly.consumed ?!

Regarding your question about alerting on the metric selector, IMO the only way is to cerate an automation that will add a data point for a custom metric with this data from beginning of the month (-1m/M to now) and set the alert on this custom metric.

HTH

Yos 

[Fin_Ubels · Accepted solution]

Since the metric only reports once a day, most of the day Dynatrace will not evaluate the metric as the anomaly detection settings can only evaluate a maximum of a 1h rolling period at a time. So what I imagine is happening is that when you go to create the alert, or test it, there hasn't been a value written to the metric for a while.

You would also need to ensure that under advanced model properties the violating samples is set to 1. By default it is set to 3 which in your case will result in no alert being fired. This is because during any given window there will only be 1 violating sample as the metric is only written once a day.

Considering the above I would check the violating samples and then also create the alert with a threshold of 1 and leave it overnight to validate.

Hopefully it's just a matter of changing the violating samples!