topic Re: Threshold Based custom alerts in Alerting

Threshold Based custom alerts

Rohan — Thu, 24 Mar 2022 08:23:20 GMT

Hello,

Thank you for taking time to look into this situation i am stuck with, I need to implement Threshold based custom alerts

The idea is :

1. CPU Usage % goes up 70% It should generate "Warning" or "Low" Severity alert.

2. CPU Usage % goes up 90% It should generate "Critical" or "High" Severity alert.

Right now based on the alert profile - As soon as CPU Usage % hits more than 90% . It ends up generating two alerts "Warning" and "Critical" both since both satisfy the condition. There's no option to implement "between 70% - 80%".

Expectation is to have One Alert triggered in an event CPU Usage % hits more than 90% and that should be "Critical" only - not two alerts.

So i seek your opinion in this matter on how this can be fixed? The Alerts are then being sent to Pager Duty just an FYI.

Regards,

Re: Threshold Based custom alerts

mrc15816 — Thu, 24 Mar 2022 02:47:20 GMT

@Rohan we went thru this hurdle as we were migrating from legacy BMC product. At this time you cannot achieve like for a like where you could generate the alerting based on threshold. I believe there is a RFE to provide some of this functionality. You can only achieve some with custom alerting ex. CPU but you can’t do that for memory as Dynatrace looks at 2 metrics (Memory usage & Swap - Composite) before triggering an alert. If you are entering into AIX you might be little bummed.

Re: Threshold Based custom alerts

Sujit — Thu, 24 Mar 2022 04:32:01 GMT

@Rohan Ideally there is no option is Dynatrace to send alert as WARNING or SEVERE, it sends alert without any priority or severity. I have no idea about PagerDuty but i have configured similar requirement in ServerNow where there is mechanism to segregate alert based on the severity through workflow.

Thanks,

Sujit

Re: Threshold Based custom alerts

DanielS — Mon, 18 Apr 2022 16:28:52 GMT

@RohanIt's not the ideal use of this, but you can try to define an SLO:

I've made an example for you, I choose CPU idle metric, you can filter the hosts by tag or entityID, and set the target to 10 and 30 (because you are using idle metric)

Then you can try it with evaluate:

I've never use this with this purpose, but it can work. Please let us know.