[k8s/oc] Downgrading OneAgent in Kubernetes and OpenShift with the Dynatrace Operator

[danmichael]

• Summary
• Problem
• Troubleshooting steps
• Resolution
• What's next

Summary

This article applies to Dynatrace Operator based OneAgent deployments on Kubernetes and OpenShift (Cloud Native Full Stack / Classic Full Stack via CSI driver). It describes how to safely downgrade the OneAgent version pinned in a DynaKube, since the Operator and its CSI driver are designed around forward upgrades and do not natively support rolling back to an older OneAgent version in place.

Problem

Dynatrace Operator manages OneAgent version updates through the DynaKube custom resource. This model works well for upgrades, but the OneAgent is not designed for downgrades.

Customers typically run into this when:

• A OneAgent version rollout (automatic or manual) introduces a regression, unexpected behavior change, or a compatibility issue with a workload, and they need to revert to the last known-good version.
• Simply editing the DynaKube to point at an older image/version tag does not fully take effect, because a newer OneAgent binary/codemodule data may already be present in the CSI driver's per-node storage or installed on the node filesystem (/var/lib/dynatrace, /opt/dynatrace, CSI data volumes), and OneAgent pods will fail to start with "downgrade is unsupported" messages.

If you are searching for this because you saw a OneAgent version mismatch across nodes after changing the pinned version in your DynaKube, or because a downgrade does not seem to apply, this article describes the supported procedure.

Troubleshooting steps

Before following the resolution below, confirm this applies to your situation:

1. Confirm you are using a Dynatrace Operator deployment, not a manually installed OneAgent.

kubectl get dynakube -n dynatrace -o yaml

2. Confirm the current OneAgent version(s) actually running, and that they differ from what you expect after editing the DynaKube:

kubectl get pods -n dynatrace -l app.kubernetes.io/name=dynatrace-oneagent -o wide

3. Check the OneAgent logs for a similar message: kubectl logs -n dynatrace ONEAGENTPODNAME

20:40:33 Logging to /var/log/dynatrace/oneagent/installer/installation_2139424.log
20:40:33 Installation started, version 1.339.71.20260706-161947, build date: 06.07.2026, PID 2139424.
20:40:33 Detected platform: LINUX arch: X86_64
20:40:33 Error: Downgrading OneAgent is not supported, please uninstall the old version first           20:40:33 Error: Attempted downgrade from 1.341.53.20260716-085120 to 1.339.71.20260706-161947

4. Confirm you have a specific, known-good previous OneAgent version to roll back to (image tag or version number), and that it is available from your image source (public ECR, your registry, or Dynatrace's download endpoint).
5. Confirm no critical workloads will be impacted by a brief loss of monitoring while OneAgent Pods are removed and recreated during the downgrade window.

Resolution

Note: Downgrading causes a short monitoring gap while the DynaKube is removed and OneAgent is reinstalled. Perform this during a maintenance window where possible.

1. Back up and remove the current DynaKube

kubectl get dynakube -n dynatrace -o yaml > dynakube.yaml
kubectl delete -f dynakube.yaml                                                                         

2. Clean up previous OneAgent installs from each node

Because OneAgent and CSI driver data can remain on nodes depending on the monitoring mode, run the node cleanup script to remove leftover data before reinstalling an older version. See Update and uninstall Dynatrace Operator — Clean up nodes and the script itself: cleanup-node-fs.sh.

The script deploys a DaemonSet that runs a cleanup procedure on all Linux nodes in the cluster (amd64, arm64, ppc64le, s390x).

Before running the node cleanup, ensure that no DynaKube is deployed and all previously monitored Pods have been restarted.

# Download the script                                                                        
curl -O https://raw.githubusercontent.com/Dynatrace/dynatrace-operator/refs/tags/v1.10.0/hack/cluster/cleanup-node-fs.sh
# Make it executable                                                                                    
chmod +x cleanup-node-fs.sh

# Run it (defaults to the "dynatrace" namespace; pass a namespace as an argument to override)           
./cleanup-node-fs.sh

The script:

• Executes the OneAgent uninstall script, if present.
• Removes OneAgent directories (/var/lib/dynatrace, /opt/dynatrace, /var/log/dynatrace).
• Removes the CSI driver data directory.
• Reports cleanup status per node.

Once all cleanup Pods complete successfully, the DaemonSet deletes itself automatically. If cleanup fails on any node, the DaemonSet remains in place for investigation — do not proceed to step 3 until this is resolved.

3. Update the DynaKube to pin the previous OneAgent version

Edit the backed-up manifest and set spec.oneAgent.cloudNativeFullStack.image (and codeModulesImage, if applicable) to the known-good version:

apiVersion: dynatrace.com/v1beta6
kind: DynaKube
metadata:
  name: dynakube
  namespace: dynatrace
spec:
  activeGate:
    capabilities:
      - routing
      - kubernetes-monitoring
      - dynatrace-api
    resources:
      limits:
        cpu: 1000m
        memory: 6Gi
      requests:
        cpu: 200m
        memory: 6Gi
  apiUrl: <YOUR_API_URL>
  metadataEnrichment:
    enabled: true
  oneAgent:
    cloudNativeFullStack:
      image: public.ecr.aws/dynatrace/dynatrace-oneagent:<PREVIOUS_VERSION>
      codeModulesImage: public.ecr.aws/dynatrace/dynatrace-codemodules:<PREVIOUS_VERSION>
      tolerations:
        - effect: NoSchedule
          key: node-role.kubernetes.io/master
          operator: Exists
        - effect: NoSchedule
          key: node-role.kubernetes.io/control-plane
          operator: Exists

Replace <YOUR_API_URL> with the apiUrl from your backed-up dynakube.yaml, and <PREVIOUS_VERSION> with the specific OneAgent build you need (e.g. 1.339.71.20260706-161947). Adjust activeGate/metadataEnrichment settings to match your original configuration.

4. Apply the updated DynaKube

kubectl apply -f dynakube.yaml

Verify the rollout:

kubectl get pods -n dynatrace -w

Confirm all OneAgent Pods come up on the pinned version and report data to your Dynatrace environment as expected.

What's next

If this article did not resolve your issue, open a support ticket, mention that this article was used, and include:

• An operator support archive. The command is only available in Dynatrace Operator version 0.11.0+. See Troubleshooting — Support archive.

kubectl exec -n dynatrace deployment/dynatrace-operator -- dynatrace-operator support-archive --stdout > operator-support-archive_$(date +%Y%m%d_%H%M).zip

Or if you don't use bash you can use the below:

kubectl exec -n dynatrace deployment/dynatrace-operator -- dynatrace-operator support-archive --stdout > operator-support-archive.zip                                                             

If you use OpenShift just replace kubectl with oc.

• The OneAgent version you downgraded from and the version you downgraded to.
• Logs from any node where the cleanup-node-fs.sh DaemonSet Pod did not complete successfully.
• A description of the regression/issue on the newer version that prompted the downgrade.

If you find yourself needing to downgrade OneAgent versions repeatedly (e.g. to test compatibility before rolling out updates broadly), consider suggesting a Product Idea describing your use case.

More articles can be found on the Kubernetes/OpenShift troubleshooting map.