Baking VM Image With ActiveGate, How To Remove From Tenant Without Uninstalling?

[ian_cervantez]

I'm using HashiCorp Packer to build gold VM images, and I'd like to build an ActiveGate base image to make it easier to bring up new synthetic locations.

I've got a working BASH script to install ActiveGate for Linux:

###
# Download Dynatrace ActiveGate
###
wget -O Dynatrace-ActiveGate-Linux-x86.sh "${DT_ENV_URL%/}/api/v1/deployment/installer/gateway/unix/latest?arch=x86&flavor=default" --header="Authorization: Api-Token $DT_PAAS_TOKEN"

###
# Install Dynatrace ActiveGate
###
chmod +x ./Dynatrace-ActiveGate-Linux-x86.sh
sudo ./Dynatrace-ActiveGate-Linux-x86.sh

###
# Simulate the uninstall to remove the ActiveGate from Dynatrace
###
sudo tee /var/lib/dynatrace/gateway/config/shutdown.info <<-'EOF'
reason = UNINSTALL
EOF
sudo chmod 0644 /var/lib/dynatrace/gateway/config/shutdown.info

###
# Shutdown ActiveGate gracefully
###
sudo systemctl stop dynatracegateway

###
# Remove the uninstall shutdown reason for future shutdowns
###
sudo rm -rf /var/lib/dynatrace/gateway/config/shutdown.info

###
# Remove the unique ID from the ActiveGate Install
###
sudo rm -rf /var/lib/dynatrace/gateway/config/id.properties

Notice that setting the reason = UNINSTALL in the shutdown.info file is the magic here.  When the ActiveGate service stops, it reads the shutdown.info file and removes the ActiveGate instance from the Dynatrace UI dashboard (removing it completely from the tenant).  After this script runs, the VM is shut down (without restarting the service), and then deleted.  The VM disk is then imaged and becomes a gold image.  Working perfectly.

I'm trying to do the same thing with Windows and PowerShell...

[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::Tls12 -bor [System.Net.SecurityProtocolType]::Tls13
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 -bor [System.Net.SecurityProtocolType]::Tls13

Write-Output "+++ Installing Dynatrace ActiveGate +++"

$programDataFolder = "$env:ProgramData\Dynatrace"
New-Item -ItemType directory -Path $programDataFolder -ErrorAction SilentlyContinue

$url      =  "$($env:DT_ENV_URL.TrimEnd('/'))/api/v1/deployment/installer/gateway/windows/latest?arch=x86&flavor=default"
$target   = "$programDataFolder\Dynatrace-ActiveGate-Windows-x86.exe"

$validExitCodes= @(0, 3010, 1641)

Write-Host "+++ Downloading Dynatrace ActiveGate +++"
Invoke-WebRequest $url -OutFile $target -Headers @{ "Authorization" = "Api-Token ${env:DT_PAAS_TOKEN}" }

Write-Host "+++ Installing Dynatrace ActiveGate +++"
$installed = Start-Process -FilePath $target -ArgumentList @("/quiet", "/qn", "/norestart", "/l*v", "$programDataFolder\Dynatrace-ActiveGate.ExeInstall.log") -Wait -Passthru

if ($validExitCodes.Contains($installed.ExitCode)) {
  Write-Host "+++ Installed Dynatrace ActiveGate +++"
}

###
# Simulate the uninstall to remove the ActiveGate from Dynatrace
###
$confPath = "$programDataFolder\gateway\config\shutdown.info"

echo "reason = UNINSTALL" | Out-File -FilePath $confPath

$ComSpec = [System.Environment]::ExpandEnvironmentVariables("%ComSpec%")
$WshShell = New-Object -ComObject WScript.Shell
$WshShell.Run("$ComSpec /C ICACLS ""$confPath"" /reset", 0)

###
# Shutdown ActiveGate gracefully
###
Stop-Service -Name "Dynatrace Gateway"

###
# Remove the uninstall shutdown reason for future shutdowns
###
Remove-Item $confPath -Force -ErrorAction SilentlyContinue

###
# Remove the unique ID from the ActiveGate Install
###
Remove-Item "$programDataFolder\gateway\config\id.properties" -Force -ErrorAction SilentlyContinue

However, it does not seem to work the same way as the Linux version of ActiveGate.  What does happen is that the shutdown.info file is deleted as the service stops.  I can only assume it's being read as the service shuts down.  However, the ActiveGate does not leave the Dynatrace dashboard UI (it's staying in the tenant).

Does anyone know more about the Windows version of ActiveGate and how to get it removed from the Dynatrace Tenant without a complete uninstall?

[Julius_Loman]

Why are you installing the ActiveGate if this is supposed to be an image? Just script only the installation itself and have the ActiveGate installed on the first run of the VM. I don't see any reason why to have a preinstalled ActiveGate. Just install all the dependencies and download and run the installer on the first run.

[ian_cervantez]

There's a few reasons, first - version pinning.  If I run the download at startup time then I get "the latest version", without testing it.  I've been burned by bugs that got introduced by Dynatrace that caused problems in our stack before so I do not want to get the latest version all the time.  Would prefer to bake a new image for each new version so we can test it.

Even if I did want the latest version all the time, we have various regulatory and compliance reasons too. We have to know that bit-for-bit what we test in lower environments is exactly what goes into production.  Installing, even a pinned version, on the fly violates this - so we'd have to get yearly exceptions granted for our audit and that can sometimes be a battle.

I could solve this by running ActiveGate in a container, but I'm not 100% sure how to get the version 1 extension modules in the container nor am I sure if running AG in a container would support memory heap dumps.  Does AG even support running a container on windows?  And, what effects does that have for the machine identity on the active directory forest? 🤔  To be honest, I don't think we're running any windows containers in our stack at all.

This would be even better solvable if I had a DELETE /activeGates/{agId} endpoint in the Environment API v2 (or config API, whatever) to delete an ActiveGate from the Dynatrace environment.

[Julius_Loman]

@ian_cervantez  You can download a specific version, not only the latest
https://www.dynatrace.com/support/help/dynatrace-api/environment-api/deployment/activegate/download-...

ActiveGate are not meant to run in container. This setup is not supported.

[ian_cervantez]

Apparently it is? https://www.dynatrace.com/support/help/setup-and-configuration/setup-on-container-platforms/kubernet...

But all the same, I don't think we could run a Windows based ActiveGate this way.

[Julius_Loman]

Sure there is, but it's for a specific purpose with only one module enabled (agent routing or Kubernetes API access). I meant it's not intended to bake it into a container by yourself. More on that here.

If you want to have a template with a fixed version, use deployment API to download the version you need and modify the settings on the first run.

[ian_cervantez]

I appreciate the responses, Julius, but this is not solving the problem. 🙂

[Julius_Loman]

@ian_cervantez  and what is the problem then? From the context I understood the version pinning - this is easily solvable by downloading the AG version you need. 

So your VM template for AG will not have AG installed in the image itself and a startup script will download and install the AG on the first run of the VM. Installation is non-interactive and takes few minutes maximum.