Dynatrace ActiveGate AutoUpdater timeout (Synthetic-enabled ActiveGate)

[Sergii]

Hi guys! Please give me advice on what happens to my ActiveGate private synthetic installation? Looks like AutoUpdater can not install something, but why?

See logs:

2023-01-05 10:55:42 UTC Dynatrace ActiveGate installer version 1.255.181.20221213-202311, architecture x86_64, build date 13.12.2022
2023-01-05 10:55:42 UTC Log initialized in /tmp/dynatracegateway-install_20230105-105542_2528.log
2023-01-05 10:55:42 UTC Checking architecture...
2023-01-05 10:55:42 UTC Architecture OK
2023-01-05 10:55:42 UTC Extracting parameters from lines 251 - 258...
2023-01-05 10:55:42 UTC After parameter extraction from the PARAMETERS section:
2023-01-05 10:55:42 UTC PARAM_SERVER_VALUE=(https://{environmentid}.live.dynatrace.com/communication,https://{environmentid}.live.dynatrace.com/communication,https://{environmentid}.live.dynatrace.com/communication,https://{environmentid}.live.dynatrace.com:443/communication)
2023-01-05 10:55:42 UTC PARAM_DEPLOYMENT_TIER_VALUE=(PROD)
2023-01-05 10:55:42 UTC PARAM_TENANT_VALUE=(*******)
2023-01-05 10:55:42 UTC PARAM_TENANT_TOKEN_VALUE=(*)
2023-01-05 10:55:42 UTC ACTIVEGATE_AUTH_TOKEN=(*)
2023-01-05 10:55:42 UTC PARAM_PROXY_VALUE=()
2023-01-05 10:55:42 UTC PARAM_PUBLIC_SG=(UNSET)
2023-01-05 10:55:42 UTC PARAM_NETWORK_ZONE=()
2023-01-05 10:55:42 UTC Processing command line parameter param (--enable-synthetic)
2023-01-05 10:55:42 UTC Dynatrace ActiveGate installer version 1.255.181.20221213-202311, build date 13.12.2022
2023-01-05 10:55:42 UTC Detected init system: 'SysV'
2023-01-05 10:55:42 UTC Checking for previous Dynatrace ActiveGate installations, verifying target installation folder...
2023-01-05 10:55:42 UTC Previous Dynatrace Gateway installation NOT detected as a running process.
2023-01-05 10:55:42 UTC No information on previous installation folders found in /etc/init.d.
2023-01-05 10:55:42 UTC Verifying system compatibility...
2023-01-05 10:55:42 UTC Reading number of CPUs... 
2023-01-05 10:55:42 UTC Checking number of CPUs... 
2023-01-05 10:55:42 UTC Number of CPUs OK
2023-01-05 10:55:42 UTC Reading RAM size... 
2023-01-05 10:55:42 UTC Checking RAM... 
2023-01-05 10:55:42 UTC RAM size OK
2023-01-05 10:55:42 UTC Reading free disk space...
2023-01-05 10:55:42 UTC System compatibility OK
2023-01-05 10:55:42 UTC Custom SELinux module will not be installed because OS is not supported.
2023-01-05 10:55:42 UTC Synthetic capability: active
2023-01-05 10:55:42 UTC Additional capabilities will be installed, if newer versions are available on the server.
2023-01-05 10:55:42 UTC Creating user: dtuserag
2023-01-05 10:55:43 UTC Checking access rights to folders...
2023-01-05 10:55:43 UTC Using custom Dynatrace Gateway user: dtuserag:dtuserag
2023-01-05 10:55:43 UTC Installing...
2023-01-05 10:55:51 UTC Installation unpacked successfully
2023-01-05 10:55:51 UTC Checking distribution in /opt/dynatrace/gateway_2528
2023-01-05 10:55:51 UTC Distribution verified
2023-01-05 10:55:51 UTC All needed folders present. Continuing.
2023-01-05 10:55:54 UTC Synthetic configuration
2023-01-05 10:55:54 UTC Will patch launcher ini with synthetic memory settings...
2023-01-05 10:55:54 UTC Setting memory in /var/lib/dynatrace/gateway/config/dynatracegateway.ini for synthetic.
2023-01-05 10:55:54 UTC Will set memory for Synthetic...
2023-01-05 10:55:54 UTC Reading RAM size... 
2023-01-05 10:55:54 UTC Updating property (/var/lib/dynatrace/gateway/config/config.properties:[collector]:deploymentTier)
2023-01-05 10:55:54 UTC Adding tenantToken to (/var/lib/dynatrace/gateway/config/authorization.properties).
2023-01-05 10:55:54 UTC Adding authToken to (/var/lib/dynatrace/gateway/config/authorization.properties).
2023-01-05 10:55:54 UTC Updating property (/var/lib/dynatrace/gateway/config/config.properties:[collector]:seedServerUrl)
2023-01-05 10:55:54 UTC Updating property (/var/lib/dynatrace/gateway/config/custom.properties:[synthetic]:synthetic_autoinstall)
2023-01-05 10:55:54 UTC Updating property (/var/lib/dynatrace/gateway/config/config.properties:[directories]:rootDir)
2023-01-05 10:55:54 UTC Updating property (/var/lib/dynatrace/gateway/config/config.properties:[directories]:packagesDir)
2023-01-05 10:55:54 UTC Updating property (/var/lib/dynatrace/gateway/config/config.properties:[directories]:tenantDataDir)
2023-01-05 10:55:54 UTC Updating property (/var/lib/dynatrace/gateway/config/config.properties:[directories]:dumpDir)
2023-01-05 10:55:54 UTC Updating property (/var/lib/dynatrace/gateway/config/config.properties:[directories]:tempDir)
2023-01-05 10:55:54 UTC Updating property (/var/lib/dynatrace/gateway/config/config.properties:[directories]:logDir)
2023-01-05 10:55:54 UTC Autobackup not found. No settings to preserve
2023-01-05 10:55:54 UTC Checking access to folders...
2023-01-05 10:55:56 UTC Updating property (/var/lib/dynatrace/gateway/config/onetime.properties:[autoupdate]:autoupdate_allowed)
2023-01-05 10:55:56 UTC Updating property (/var/lib/dynatrace/gateway/config/onetime.properties:[autoupdate]:autoupdate_enabled)
2023-01-05 10:55:56 UTC Updating property (/var/lib/dynatrace/gateway/config/onetime.properties:[autoupdate]:autoupdate_activegate)
2023-01-05 10:55:56 UTC Updating property (/var/lib/dynatrace/gateway/config/onetime.properties:[collector]:allowedModules)
2023-01-05 10:55:56 UTC Updating property (/var/lib/dynatrace/gateway/config/onetime.properties:[autoupdate]:force_update)
2023-01-05 10:55:56 UTC One-time module downloading has been activated
2023-01-05 10:55:56 UTC Fresh installation of Dynatrace ActiveGate. Removing old configuration files of installed capabilities.
2023-01-05 10:55:56 UTC Starting Dynatrace Gateway...
05/01 10:55:56 (1.255.181.20221213-202311) Dynatrace ActiveGate started.
05/01 10:55:56 (1.255.181.20221213-202311) 
2023-01-05 10:56:15 UTC Successfully connected to server https://{environmentid}.live.dynatrace.com/communication,https://{environmentid}.live.dynatrace.com/communication,https://{environmentid}.live.dynatrace.com/communication,https://{environmentid}.live.dynatrace.com:443/communication
2023-01-05 10:56:15 UTC Starting Dynatrace ActiveGate AutoUpdater...
05/01 10:56:15 Dynatrace ActiveGate AutoUpdater 1.255.181.20221213-202311 started.
2023-01-05 10:56:20 UTC Checking if Dynatrace ActiveGate AutoUpdater is running ...
2023-01-05 10:56:20 UTC Dynatrace ActiveGate AutoUpdater is running.
2023-01-05 10:56:20 UTC Cleaning autobackup...
2023-01-05 10:56:20 UTC Waiting for modules download. Timeout in 15 minutes.
2023-01-05 10:57:20 UTC Waiting for modules download. Timeout in 14 minutes.
2023-01-05 10:58:20 UTC Waiting for modules download. Timeout in 13 minutes.
2023-01-05 10:59:20 UTC Waiting for modules download. Timeout in 12 minutes.
2023-01-05 11:00:20 UTC Waiting for modules download. Timeout in 11 minutes.
2023-01-05 11:01:20 UTC Waiting for modules download. Timeout in 10 minutes.
2023-01-05 11:02:20 UTC Waiting for modules download. Timeout in 9 minutes.
2023-01-05 11:03:20 UTC Waiting for modules download. Timeout in 8 minutes.
2023-01-05 11:04:20 UTC Waiting for modules download. Timeout in 7 minutes.
2023-01-05 11:05:20 UTC Waiting for modules download. Timeout in 6 minutes.
2023-01-05 11:06:20 UTC Waiting for modules download. Timeout in 5 minutes.
2023-01-05 11:07:20 UTC Waiting for modules download. Timeout in 4 minutes.
2023-01-05 11:08:20 UTC Waiting for modules download. Timeout in 3 minutes.
2023-01-05 11:09:20 UTC Waiting for modules download. Timeout in 2 minutes.
2023-01-05 11:10:20 UTC Waiting for modules download. Timeout in 1 minutes.
2023-01-05 11:11:20 UTC WARNING: Timeout while downloading capabilities.
2023-01-05 11:11:20 UTC Stopping Dynatrace Gateway...
05/01 11:11:20 (1.255.181.20221213-202311) Terminating Dynatrace ActiveGate. Process id: 5063
05/01 11:11:33 (1.255.181.20221213-202311) 
2023-01-05 11:11:33 UTC 
2023-01-05 11:11:33 UTC --------------------------------------------------------------
2023-01-05 11:11:33 UTC Removing old installation log files...
2023-01-05 11:11:33 UTC Installation finished successfully.

[ankurs]

@Sergii I am facing a similar issue while installing environment activegate. have you found any solution to your problem?

[Sergii]

Hi @ankurs , I did not managed to find a solution. But I guess this issue is related to some kind of "not enough resources" stuff. The point why I think so is that I tried to install Synthetic-enabled ActiveGate in minikube on ubuntu container. Probably minikube does not provide enough memory or CPU. Later I installed the same type of ActiveGate to my host windows mashine and it was ok, so maybe it will help you.

[HannahM · Accepted solution]

Synthetic AGs are not currently supported for installation in containers, so possibly that is causing this behaviour. If you just install it on Ubuntu, do you see the same issue?

[ankurs]

@HannahM  I am facing this issue while installing environment AG on a Linux machine. any idea about that behavior? 

[HannahM]

So you see the same 

WARNING: Timeout while downloading capabilities.

message in the gateway installation log?
Can you confirm that the AG can successfully connect to the cluster? This is where it will download the Synthetic Module from
Can you also check the 

/var/lib/dynatrace/packages

 folder. Are there any .sh files in there? Specifically 

Dynatrace-Synthetic-Linux-x86-*

files. If so, what size are they? 0kb? If so, usually this occurs when a virus scanner removes it. 

[ankurs]

Yes, I see the same message

WARNING: Timeout while downloading capabilities.

AG can successfully connect to the cluster as I can see below log

2023-01-30 14:32:12 UTC Starting Dynatrace Gateway...
2023-01-30 14:32:27 UTC Successfully connected to server

And failing after the timeout error mentioned in the description. For me below directory is empty 

/var/lib/dynatrace/packages

and there is no .sh file. Are you talking about the file something similar to 

Dynatrace-ActiveGate-Linux-x86-1.257.216.sh

For me, this file is present in the root folder.

[HannahM]

Thanks for checking. The Dynatrace-ActiveGate-Linux-x86-1.257.216.sh is the ActiveGate installer. This then downloads the relevant modules - Synthetic, remote etc to the packages folder. It seems it's the Synthetic module that is not able to download. It would be easiest to investigate this by checking the logs for the Activegate. Can you start a chat or create a ticket? Then we can check exactly what is happening on this machine. Thanks

[ankurs]

Sure @HannahM . Let me do the needful. Thanks for your help.

[DanielS]

Please, can you check if you have connection to https://synthetic-packages.s3.amazonaws.com

?

[DanielS]

If you do not have connectivity, it is very likely that the update will fail.

[ankurs]

@DanielS I did curl on the above URL and below was the result

# curl https://synthetic-packages.s3.amazonaws.com
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>CXYKZYQ11K8SAR9X</RequestId><HostId>B2AkY/OFkqXXLgDLLjQvrC5qj0RZEqbi+/p7C0HuKiy4jLeJlXpLZtCM5irdfl+yH86tKe9ti/U=</HostId></Error>[root@alldstestdt2-1 ~]# 

 is it an unexpected output?

[DanielS]

Yes, an Access Denied here implies connectivity, because to get the Denied you first need to reach the resource.

To test try with this. curl --output chromium.tgz https://synthetic-packages.s3.amazonaws.com/Chromium/rpm/chromium-103.0.5060.114-1.el7.tgz

[ankurs]

No connectivity issue.

[root@alldstestdt2-1 ~]# curl --output chromium.tgz https://synthetic-packages.s3.amazonaws.com/Chromium/rpm/chromium-103.0.5060.114-1.el7.tgz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 98.0M  100 98.0M    0     0  5260k      0  0:00:19  0:00:19 --:--:-- 12.5M

[DanielS]

Ok, you can discard that, let's wait to see with support what else could be happening. Please let us know.