Host State and Alerting

[jarinmusarrat]

Hi team,

My client shutdown about 13 hosts in the prod tenant and 2 of them marked as offline and created an alert. But the other 11 were on shutdown state and didn't create any alert. From the documentation (https://www.dynatrace.com/support/help/how-to-use-dynatrace/hosts/monitoring/host-availability#offli...) I see the explanation that if the host gets interrupted during shutdown and not graceful then it creates alerts. But customer asking how do they determine if it is an actual problem or not? since they shut them all down the same way what or how is Dynatrace determining those 2 alerts but found the other 11 hosts to have graceful shutdown? 

[techean]

this might happen due to agents connectivity with Dynatrace. Basically in this case the agent may not able to send the actual data due to connectivity while graceful shut down event was trigger. one agent logs should be able to provide more insights on same.

[jarinmusarrat]

@techean are you saying that the fact that agent may not be able to send the actual data due to connectivity can happen by chance? And if so how would it be possible to distinguish it from an actual problem to this? Also which log can I look into if the host is already in shutdown or unavailable mode would it still be able to collect log data?

[techean]

You could refer chad comments, but as you said the host were turned off as the rest, than this issue can only be available on those host path of oneagent where logs are stored.

/opt/dynatrace/oneagent

[ChadTurner · Accepted solution]

The system will review the drop in communication and classify it as either a shutdown of the host or a loss of communication. If a host is shut down the the oneagent would have seen the shutdown command come through and therefore either alert or not alert based off of the anomaly settings. If the host was shut down abruptly then it will classify it as a shutdown but not graceful and then it will trigger an alert. Lastly if the oneagent stops communicating, then the system will mark it as a connection problem and will alert you on that as well.