topic How to alert on usually monitored services that become unmonitored? in Alerting

How to alert on usually monitored services that become unmonitored?

vlazarova — Mon, 22 Aug 2022 14:16:25 GMT

Hi,

We are monitoring certain application services on Windows hosts, and we receive alerts when they (the services) are not running for some reason. However, we don't receive alerts when they become "unmonitored". How can this be set? Is it through "process availability" for the whole process group?

Example of the issue:

1. Service "X" is being monitored by Dynatrace

2. During Windows patching service "X" is stopped, the server is rebooted, etc.

3. After the server reboot the service doesn't come up, because it has "Manual" as startup type

4. An hour or two after that the server crashes unexpectedly and reboots itself again. Service "X" doesn't come up again (set to "Manual")

5. Since the server crash Dynatrace shows service "X" as "unmonitored" rather than "shutdown". The service started showing as running (respectively monitored) only after it was manually started on the server.

So, I guess my question is if there's a way for Dynatrace to raise a problem once a usually monitored services becomes unmonitored and how that can be achieved.

Thanks in advance!

Re: How to alert on usually monitored services that become unmonitored?

mark_bley — Wed, 31 Aug 2022 20:29:42 GMT

Hi @vlazarova

there is a new feature you can find under Settings > OS service monitoring

Here you can define a rule specific to that service or e.g. in general all Manual Windows OS Services like follows:

 "value": {
        "enabled": true,
        "system": "WINDOWS",
        "name": "Manual Windows OS Services",
        "monitoring": true,
        "alerting": true,
        "statusConditionWindows": "$not($eq(running))",
        "detectionConditionsWindows": [
          {
            "property": "StartupType",
            "startupCondition": "$eq(manual)"
          }
        ],
        "metadata": []
      }

Or in the UI it would look like this

Feel free to add more rules e.g. for a specific service only you may adda rule for its name.

Then in terms of alerting you can play around with alerting profiles if you don't want to get notified instantly in case the service takes longer to come back

Best,

Mark

Re: How to alert on usually monitored services that become unmonitored?

Malaik — Thu, 11 May 2023 13:07:44 GMT

Hi @mark_bley

Do you know if this option help in case if the Host is OFF, Shutdown, or the agent removed....

Re: How to alert on usually monitored services that become unmonitored?

mark_bley — Tue, 16 May 2023 11:06:19 GMT

no, but you can trigger problems if the host is gracefully shut down

This setting is available on environment, host group or host level.

Found under Settings > Anomaly detection > Hosts

Re: How to alert on usually monitored services that become unmonitored?

sundarv1 — Fri, 24 May 2024 03:52:06 GMT

Currently OS Service monitoring incidents are going to default profile to availability and hence creating as P2.

How do we move this different profile so that it creates P3 incident.

Re: How to alert on usually monitored services that become unmonitored?

AntonPineiro — Fri, 24 May 2024 16:05:15 GMT

Hi,

You need to add/modify rules in those alerging profiles, base on OS service metadata.

Best regards

Re: How to alert on usually monitored services that become unmonitored?

sundarv1 — Fri, 24 May 2024 16:13:32 GMT

Could you please share steps with screenshot how to setup and generate slowdown alert

Re: How to alert on usually monitored services that become unmonitored?

mark_bley — Fri, 24 May 2024 16:16:00 GMT

You can add event properties, description or title to filter by in your os:service config in order to use these to filter in the alerting profile

os:service config sample:

alerting profile sample:

Re: How to alert on usually monitored services that become unmonitored?

sundarv1 — Fri, 24 May 2024 17:20:44 GMT

My request is created a slowdown event for OS service issues. Current it is creating as Availablity event. How to do this

Re: How to alert on usually monitored services that become unmonitored?

mark_bley — Mon, 27 May 2024 08:42:58 GMT

Not possible to change the eventType property.

os:service config will throw an AVAILABILITY_EVENT, please use any other type of event property and filter by that in your alerting profile and route your tickets accordingly

Re: How to alert on usually monitored services that become unmonitored?

sundarv1 — Mon, 27 May 2024 08:54:22 GMT

Thanks . How do we change when OS service event comes out of dynatrace - it become a slowdown event , not as availability event.