Sometimes we receive an email notification from DC RUM with delay. Today, the alert started at 1:20 am but we received the email notification at 2:08 am. Could it be because of delay in data processing in that moment?
Alert log contains the following:
Tue May 24 01:20:00 CEST 2016•LOW_OPER_4_SYS_MOD•Alert started•Inoperability detected for reporting server module: Data Processing, sub-module: Delay. Inoperability cause: Delay in performance data processing is 38 minute(s). Report on system status: Reference URL: /modulestatus•1464045600000•Error•Data Processing•Delay•Delay in performance data processing is 38 minute(s)
Tue May 24 01:20:00 CEST 2016•TRANSMETRIC_ALM_10009•Alert started•..........
Solved! Go to Solution.
You are right in identifying that the delay in email notifications is tied to the delay in data processing. After a given monitoring interval has passed, the CAS will collect and process the data files from the AMDs. Once this is done, a number of other tasks are done to complete the processing for that interval, including evaluating whether alerts are triggered. If the processing of the interval data is delayed, the alert evaluations are also delayed.
Because the delay you mentioned is between 1 am and 2 am, this delay is likely to be a result of the nightly tasks being run. These nightly tasks are necessary to maintain the DCRUM system health and include such activities as aggregating data, daily database maintenance and running daily scheduled reports. It is necessary for the CAS to stop processing monitoring data until the nightly tasks are completed.
Thanks for your reply but I still do not understand why there is almost 1 hour between generate an alert and send an email. I understand that generating of alert could be delayed because of the nightly tasks but I suppose sending email notification should be part of alert task without delay.
Maybe I am not correct about how notification workflow works.