topic Re: What situation does watchdog cover on Managed Server? in Dynatrace Managed Q&A

What situation does watchdog cover on Managed Server?

yohnishi — Tue, 27 Oct 2020 00:51:02 GMT

Hi,

We are installing and testing the Managed Server for Production environment.

So we are killing some processes in the Managed Server in anticipation of a process failure, but the watchdog doesn't seem to be working and the process is not being restarted. Is this a specification?

We killed the following process with "kill -9 <PID>", based on the PIDs available in "dynatrace.sh status".

- activegate

- nginx

- elasticsearch

Under what situations would nodekeeper or watchdog work to restart the process and keep it healthy?

Best Regards,
Yasuo Ohnishi

Re: What situation does watchdog cover on Managed Server?

Radoslaw_Szulgo — Fri, 09 Apr 2021 12:41:34 GMT

Watchdog checks every 1 minute if its main process (Cassandra, Elasticsearch, Security-gateway (ActiveGate), Nodekeeper) is running. If it is not, then the restarts is triggered.

Nginx is a bit different here. It consists of a "primary" process and "secondary" processes. Primary is kind like a watchdog here. If you kill the primary process, nothing will happen and it won't start automatically. If you kill a secondary process, a new one should be spawned by the primary process. Anyway, we haven't observed in our production systems primary process crashes.

I'm not sure what specifically test you have executed, but it's important to understand what processes you kill to know the expectation.

What is more, you should get some alerting visible in CMC -> Events, if a processes is down.

Re: What situation does watchdog cover on Managed Server?

yohnishi — Tue, 27 Oct 2020 14:13:56 GMT

Hi, Radoslaw-san,

Thank you for your replay.

I killed the process of the "Main PID", so I think I killed the watchdog side. I'll check again.

Best Regards,
Yasuo Ohnishi