Process Crashed Alerting

Platform Alerting
1

I would like to create an alert for when a process has crashed.
Has anyone done something similar?

2

Does Process group availability monitoring and alerting work?

3

Yes it does, granted you need to have the following set:

ChadTurner_0-1638927985778.png
ChadTurner_0-1638927985778.png947×561 31.6 KB

And then the associated Alert Profile will trigger then trigger the alert integration and send out an alert as you have set up.

4

Doesn’t help in a scenario where there are multiple worker processes and we want to alert when any of the workers crash.   We can see it in DT, availability does not help here.  It would be nice to separate availability events caused by crashes anyway.

5

Hello,

This can be achieved with Event API, via a developed script to make an Event API call periodically and limited it to only crash events. In my environment I am monitoring a large scale of servers with more than 600 applications, most of the crashed events actually come from unimportant processes which only executed at ad hoc basis. To filter out the noise, I will proceed further to verify the crashed process uptime using the time-series CPU usage metric to filter out the false alarm.

Please refer to the attached picture below to understand how it works, hope it helps.

Crash Monitoring.png
Crash Monitoring.png1535×750 45.4 KB

6

In our use-case we have Processes which are not running all the time, to it’s normal that they become “unavailable” by design.
On the Processes->RandomProcess.exe Page there is this beautiful “Events” Graph which would be nice if we could use them for custom metrics. Or from the Graph in the “Application & Microservices” - “Profiling and optimization” - “Crashes” section.

7

Alarm threshold based on number of worker processes

8

Would be nice if you could alert on the crash events, as someone here posted. Anyone been able to solve this?

9

Can you share the event as it shows up for your crash. You can create a custom Event to trigger off an alert when a crash is reported.

We did something similar for Pods that went into a ‘Killing’ State.

10

Hi,

Kindly provide more details on leveraging Event API to address this issue. We want to send a email notification to team whenever the Process crashes.

Thanks !