topic Re: Problem notification delivery verification in Alerting

Problem notification delivery verification

kristof_renders — Wed, 03 Jan 2024 13:06:49 GMT

Hi,

What are the retry policies for problem notification?

Customer wants to know how Dynatrace handles the external system not being there or returning for example a HTTP 500. Will it retry and if so how many times? Will the failure be logged somewhere and can be (manually retried)?

Thanks in advance.

KR,
Kristof

Re: Problem notification delivery verification

wolfgang_beer — Fri, 24 Nov 2017 13:12:43 GMT

We trigger a retry two times every 15min. We get a log message but there is no indicator for the users so far. We will try to give a feedback in the future. There is no way of trigger a manual resend.

Re: Problem notification delivery verification

kristof_renders — Fri, 24 Nov 2017 13:58:33 GMT

Thanks Wolfgang.

The customer is concerned that there will be problems that go by unnoticed. Already good to know that there are two retries. However if both of those fail it should notify the admin and also offer the possibility to manually fire it again. Best to create an RFE for this?

KR,
Kristof

Re: Problem notification delivery verification

chris_v — Tue, 18 May 2021 02:36:44 GMT

We definitely need a notification within the CMC (for managed)/environment when notifications fail to be sent.

I've just run into a problem (today) where webhook notifications to PagerDuty have been failing since April 25th, we had no idea it wasn't working. The customer raised the concern that there were no notifications for over a week. found the log entries on the cluster, but that's on;y useful after the fact triaging when the problem started.