Is there a way to gracefully shut down an ActiveGate?

[ian_cervantez1]

Based on the documentation here:

https://www.dynatrace.com/support/help/setup-and-configuration/dynatrace-activegate/operation/stop-r...

When shutting down an ActiveGate (to apply an update, or to remove it and add a new one in its place), the only mechanism is to stop the service. This bothers me, since it isn't a graceful shut down but rather an abrupt stop on the process. The ActiveGate may be in the middle of a synthetic test when the shutdown happens. Or, it may have collected OneAgent metrics and hasn't sent the metrics along to Dynatrace yet.

Is there a way to safely drain traffic from the ActiveGate prior to shut down? Leave the ActiveGate online, but stop it from collecting new metrics from OneAgents and discontinue running synthetic monitoring tests?

More importantly, once the traffic has drained properly, can I automatically trigger the stop on the ActiveGate?

[ctturner]

Based on my experience from running updates automatically and manually, We use the stop command as it starts to shutdown the services and isn't a true abrupt stop which can be used if you call for a 'force stop' The OneAgents will retain the metrics for about 60 minutes until the activegate comes back online after the update. When it comes to the synthetics as we make them use dedicated activegates, and when we update it essentially puts the test on a hold. Our Synthetics run every 30 minutes and when I perform a long maintenance on the activegate for synthetics we don't get alerted that synthetics failed because I used the stop command and it gracefully shutdown the service.

[ian_cervantez1]

That's great news about the OneAgents holding metrics for 60 minutes. I've got a multi-activegate setup, so if the OneAgents find a different ActiveGate that's online to send metrics to - it'll use the new one automagically right?

As for the Synthetics, I'm not worried about getting alerts for failure to run. I'll have new ActiveGates online to replace the old ActiveGates I'm shutting down. I'm worried about gaps in the synthetic coverage caused by shutting down an ActiveGate in the middle of a synthetic test. If the currently running synthetic test is allowed to finish before stop kills the ActiveGate, then I'm all set. Let's walk through a scenario:

1. A particularly long synthetic test with click paths may take 5-10 minutes to perform.
2. Start this long synthetic test.
3. 2 minutes into the synthetic test, the stop command is sent to the service.

Does the service respect the running synthetic test and wait the remaining 3-8 minutes before shutting down the service?

[ctturner]

After running a few tests I can confirm that the stop command will end any active synthetics tests.

[ian_cervantez1]

Oh, bummer. 😞

[ian_cervantez1]

I've submitted a product idea for this:

https://community.dynatrace.com/spaces/483/dynatrace-product-ideas/idea/224503/schedule-activegate-g...

[MaciejNeumann · Accepted solution]

Comment from Synthetic Monitoring Product Manager:

"Active gate shutdown caused by update requests is graceful from executing synthetic tests point of view. There is timeout (now 5 minutes, will be increased to 7,5 minutes in 192) which is longer than the longest timeout for executing single test (Browser clickpath, 5 minutes). The procedure uses this timeout to wait for all currently executed tests to be finished. Such approach assures, that all tests which are executed when upgrade requests appears, will be finished. What you can observer is only slight delay in starting next tests caused by timeout and upgrade time

Please, notice, that it is internal upgrade procedure which doesn’t cover scenario of manual restart of machine on which Active Gates is running. In such case we have to base on standard system timeouts, which might be shorter."

[skrystosik]

Agents are holding metrics little bit shorter that you've said. It also depends of amount of traffic on particular host 🙂

[ian_cervantez1]

Do you know how much "a little bit shorter" is? Are we talking, like, holding metrics for 30 minutes? Or is it substantially shorter like 1 minute?

[ctturner]

From the CMC side you can also select to disable or restart the activegate from the CMC console. This will start that shutdown procedure and not do an abrupt stop. The only difference between the two options is that one will turn the services back on, just like restarting your computer, it safely closes everything, then turns it all back on.