HI,
We observed 15 min delay in problem email notification. But the teams expect the alert notification at real time. Kindly let me know the reason behind this delay and if there is any workaround.
HI,
We observed 15 min delay in problem email notification. But the teams expect the alert notification at real time. Kindly let me know the reason behind this delay and if there is any workaround.
@ted d. Can you confirm that the alerting profile is set to alert immediately of the issue? and not âXâ amount of mins. after? Also the system might have a 1-5 min delay of the email arriving in the users inbox. To combat this, we also incorporate the Dynatrace UFO and Mobile APP Alert.
Hi @ted d. in first place is important to understand that Dynatrace is not âjust another silly alerting toolâ. When an event took place the IA or Davies is doing a huge correlation between all your components. And alerts you when all the analysis took place, this is not in realtime but of course it can not take 5â as you described.
In my experience with several clients this could be due to an incorrect sizing of the underlying hardware where Dynatrace runs. (Of course this applies to Dynatrace Managed only) Please check the requirements at https://www.dynatrace.com/support/help/setup-and-configuration/dynatrace-managed/installation/dynatrace-managed-hardware-and-system-requirements/
Dynatrace Managed operates on all common file systems. The recommendation is that you select fast local storage appropriate for database workloads. High latency remote volumes like NFS or CIFS arenât recommended. While NFS file systems are sufficient for backup purposes, itâs not recommended for primary storage.
If SaaS is your case, is important to check the bandwith between ActiveGate and SaaS is not suffering from starvation due to âall the other trafficâ. Depending of your firewall, you can prioritize this, also check hardware requirements for your ActiveGate, please check it at Installation â Dynatrace Docs
At last when we take care of this, the delays you described were kept to a maximum of 2â for complex cases on environments > 500 Hosts Units and near real time in smaller installations.
Chad,
Do you see differences between emails and the app? They fall in my case almost exactly immediately.
Just to add that the person who I talked to said that they were looking into this carefullyâŚ
we found the App to be 1 second behind the UI raising a problem card. same with the UFO - for a managed stand point with 3 clustered nodes
Thought it was a lot more. In my case I have not analyzed it with that precision ![]()
Do you guys use the ufo as well?
No, not yetâŚ
I would recommend it
How UFO will help in this case
The ufo will visually alert you to an issue in your infrastructure via the color codes. We have 5 deployed, and people love it.
Hi Daniel
we saw a delay of 56 min in receiving the email the problem started at 13:01 and the customer got an email notification at 13:57 can that be network issue or any thing else?
Thanks
Taneshaa
Please check the email headers to see where it might have got delayed. Also check itâs not a frequent issue, because in these cases, the alert might be delayed.
Hi @ThaplooT
In addition to @AntonioSousa reply you can check dsfm:server.notifications.problem_notifications & dsfm:server.notifications.problem_notifications_duration metrics which will give you the cluster perspective on when cluster fire the notification with status and timings
HTH
Yos