Baking VM Image With ActiveGate, How To Remove From Tenant Without Uninstalling?

[ian_cervantez]

I'm using HashiCorp Packer to build gold VM images, and I'd like to build an ActiveGate base image to make it easier to bring up new synthetic locations.

I've got a working BASH script to install ActiveGate for Linux:

###
# Download Dynatrace ActiveGate
###
wget -O Dynatrace-ActiveGate-Linux-x86.sh "${DT_ENV_URL%/}/api/v1/deployment/installer/gateway/unix/latest?arch=x86&flavor=default" --header="Authorization: Api-Token $DT_PAAS_TOKEN"

###
# Install Dynatrace ActiveGate
###
chmod +x ./Dynatrace-ActiveGate-Linux-x86.sh
sudo ./Dynatrace-ActiveGate-Linux-x86.sh

###
# Simulate the uninstall to remove the ActiveGate from Dynatrace
###
sudo tee /var/lib/dynatrace/gateway/config/shutdown.info <<-'EOF'
reason = UNINSTALL
EOF
sudo chmod 0644 /var/lib/dynatrace/gateway/config/shutdown.info

###
# Shutdown ActiveGate gracefully
###
sudo systemctl stop dynatracegateway

###
# Remove the uninstall shutdown reason for future shutdowns
###
sudo rm -rf /var/lib/dynatrace/gateway/config/shutdown.info

###
# Remove the unique ID from the ActiveGate Install
###
sudo rm -rf /var/lib/dynatrace/gateway/config/id.properties

Notice that setting the reason = UNINSTALL in the shutdown.info file is the magic here.  When the ActiveGate service stops, it reads the shutdown.info file and removes the ActiveGate instance from the Dynatrace UI dashboard (removing it completely from the tenant).  After this script runs, the VM is shut down (without restarting the service), and then deleted.  The VM disk is then imaged and becomes a gold image.  Working perfectly.

I'm trying to do the same thing with Windows and PowerShell...

[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::Tls12 -bor [System.Net.SecurityProtocolType]::Tls13
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 -bor [System.Net.SecurityProtocolType]::Tls13

Write-Output "+++ Installing Dynatrace ActiveGate +++"

$programDataFolder = "$env:ProgramData\Dynatrace"
New-Item -ItemType directory -Path $programDataFolder -ErrorAction SilentlyContinue

$url      =  "$($env:DT_ENV_URL.TrimEnd('/'))/api/v1/deployment/installer/gateway/windows/latest?arch=x86&flavor=default"
$target   = "$programDataFolder\Dynatrace-ActiveGate-Windows-x86.exe"

$validExitCodes= @(0, 3010, 1641)

Write-Host "+++ Downloading Dynatrace ActiveGate +++"
Invoke-WebRequest $url -OutFile $target -Headers @{ "Authorization" = "Api-Token ${env:DT_PAAS_TOKEN}" }

Write-Host "+++ Installing Dynatrace ActiveGate +++"
$installed = Start-Process -FilePath $target -ArgumentList @("/quiet", "/qn", "/norestart", "/l*v", "$programDataFolder\Dynatrace-ActiveGate.ExeInstall.log") -Wait -Passthru

if ($validExitCodes.Contains($installed.ExitCode)) {
  Write-Host "+++ Installed Dynatrace ActiveGate +++"
}

###
# Simulate the uninstall to remove the ActiveGate from Dynatrace
###
$confPath = "$programDataFolder\gateway\config\shutdown.info"

echo "reason = UNINSTALL" | Out-File -FilePath $confPath

$ComSpec = [System.Environment]::ExpandEnvironmentVariables("%ComSpec%")
$WshShell = New-Object -ComObject WScript.Shell
$WshShell.Run("$ComSpec /C ICACLS ""$confPath"" /reset", 0)

###
# Shutdown ActiveGate gracefully
###
Stop-Service -Name "Dynatrace Gateway"

###
# Remove the uninstall shutdown reason for future shutdowns
###
Remove-Item $confPath -Force -ErrorAction SilentlyContinue

###
# Remove the unique ID from the ActiveGate Install
###
Remove-Item "$programDataFolder\gateway\config\id.properties" -Force -ErrorAction SilentlyContinue

However, it does not seem to work the same way as the Linux version of ActiveGate.  What does happen is that the shutdown.info file is deleted as the service stops.  I can only assume it's being read as the service shuts down.  However, the ActiveGate does not leave the Dynatrace dashboard UI (it's staying in the tenant).

Does anyone know more about the Windows version of ActiveGate and how to get it removed from the Dynatrace Tenant without a complete uninstall?

[Julius_Loman]

Why are you installing the ActiveGate if this is supposed to be an image? Just script only the installation itself and have the ActiveGate installed on the first run of the VM. I don't see any reason why to have a preinstalled ActiveGate. Just install all the dependencies and download and run the installer on the first run.

[ian_cervantez]

There's a few reasons, first - version pinning.  If I run the download at startup time then I get "the latest version", without testing it.  I've been burned by bugs that got introduced by Dynatrace that caused problems in our stack before so I do not want to get the latest version all the time.  Would prefer to bake a new image for each new version so we can test it.

Even if I did want the latest version all the time, we have various regulatory and compliance reasons too. We have to know that bit-for-bit what we test in lower environments is exactly what goes into production.  Installing, even a pinned version, on the fly violates this - so we'd have to get yearly exceptions granted for our audit and that can sometimes be a battle.

I could solve this by running ActiveGate in a container, but I'm not 100% sure how to get the version 1 extension modules in the container nor am I sure if running AG in a container would support memory heap dumps.  Does AG even support running a container on windows?  And, what effects does that have for the machine identity on the active directory forest? 🤔  To be honest, I don't think we're running any windows containers in our stack at all.

This would be even better solvable if I had a DELETE /activeGates/{agId} endpoint in the Environment API v2 (or config API, whatever) to delete an ActiveGate from the Dynatrace environment.

[Julius_Loman]

@ian_cervantez  You can download a specific version, not only the latest
https://www.dynatrace.com/support/help/dynatrace-api/environment-api/deployment/activegate/download-...

ActiveGate are not meant to run in container. This setup is not supported.

[ian_cervantez]

Apparently it is? https://www.dynatrace.com/support/help/setup-and-configuration/setup-on-container-platforms/kubernet...

But all the same, I don't think we could run a Windows based ActiveGate this way.

[Julius_Loman]

Sure there is, but it's for a specific purpose with only one module enabled (agent routing or Kubernetes API access). I meant it's not intended to bake it into a container by yourself. More on that here.

If you want to have a template with a fixed version, use deployment API to download the version you need and modify the settings on the first run.

[ian_cervantez]

I appreciate the responses, Julius, but this is not solving the problem. 🙂

[Julius_Loman]

@ian_cervantez  and what is the problem then? From the context I understood the version pinning - this is easily solvable by downloading the AG version you need. 

So your VM template for AG will not have AG installed in the image itself and a startup script will download and install the AG on the first run of the VM. Installation is non-interactive and takes few minutes maximum.

[KevinW]

I know this was/is an old thread, but I'm trying to automate the same Deployment/Uninstallation process, and noticed the same, that even after uninstall, the Windows based ActiveGate appears to simply be marked as "inactive" in Console.

I noticed you just stop the some of the Services (and not all of them) - ActiveGate non-synthetic has 4 Services:

get-service -Name *dyn*

Status Name DisplayName 
------ ---- ----------- 
Running Dynatrace Autou... Dynatrace Autoupdater 
Running Dynatrace Exten... Dynatrace Extensions Controller 
Running Dynatrace Gateway Dynatrace Gateway 
Running Dynatrace Remot... Dynatrace Remote Plugin Module

Assuming standard locations, then the  powershell "uninstall" of the ActiveGate would be:

get-package -name *dyn* | Uninstall-Package -Force

Then there is the log and file clean up:

Remove-Item -Recurse $Env:ProgramData\dynatrace\gateway\log
Remove-Item -Recurse $Env:ProgramData\dynatrace\packages
Remove-Item -Recurse $Env:ProgramData\dynatrace\autoupdater
Remove-Item -Recurse $Env:ProgramData\dynatrace\remotepluginmodule

The ActiveGate remains present in the Console until the some internal timeout occurs to trigger the clean-up. I've had this "historic" Windows ActiveGate remain for several days.

If the AG is re-installs on the same host/IP address, then it re-activates that "inactive" ActiveGate record.

[KevinW]

I should add, that if I RDP to the Windows ActiveGate and trigger the "software uninstall" via the GUI, then the Console DOES update immediately, so perhaps stopping the services is causing the feedback to the Console not to be sent.

So I changed the above order to simply run the Package uninstall (without stopping the services) and the console did empty immediately.

Still not convinced this is completely unattended installation yet, as I stll had a popup window.