Kubernetes/Openshift troubleshooting map

[shahinm]

Getting started

• Main documentation page of Set up Dynatrace on Kubernetes
• Kubernetes on Is It Observable?

Product architecture

• Classic full-stack injection
• Cloud native full-stack injection
• Application-only monitoring
• Host monitoring
• Kubernetes Platform Monitoring
• Kubernetes Security Posture Management (KSPM)

Product usage

• Official GitHub repository:
  • Dynatrace Operator 
  • Samples of DynaKube CR
• Guides to deploy OneAgent in various modes:
  • Kubernetes platform monitoring
  • Application observability
  • Cloud Native Full-Stack observability
  • Classic Full-Stack Deployment
  • Host Monitoring
  • Application observability with pod runtime injection
  • Application observability via container build-time injection
  • Kubernetes Security Posture Management

Troubleshooting

• Monitoring issues and troubleshooting
  • Pods stuck in Terminating state after upgrade
  • Unable to retrieve the complete list of server APIs
  • CrashLoopBackOff: Downgrading OneAgent is not supported, please uninstall the old version first
  • Crash loop on pods when installing OneAgent
  • Deployment seems successful but the dynatrace-oneagent container doesn't show up as ready
  • Deployment seems successful, however the dynatrace-oneagent image can't be pulled
  • Deployment seems successful, but the dynatrace-oneagent container doesn't produce meaningful logs
  • Deployment seems successful, but the dynatrace-oneagent container isn't running
  • Deployment was successful, but monitoring data isn't available in Dynatrace
  • No pods scheduled on control-plane nodes
  • Error when applying the custom resource on GKE
  • CannotPullContainerError
  • Limit log timeframe
  • Dynatrace Kubernetes service creation fails when Istio is enabled
  • Application pods are stuck in terminating state
  • Application pods can't start post injection
  • AKS WASM node-pools troubleshooting
  • Cloud Native Full Stack - Pod Injection Validation and Troubleshooting
• Connectivity issues between Dynatrace and Kubernetes cluster
  • Problem with ActiveGate token
  • ImagePullBackoff error on OneAgent and ActiveGate pods
  • There was an error with the TLS handshake
  • Invalid bearer token
  • Could not check credentials. Process is started by other user
  • Internal error occurred: failed calling webhook (…) x509: certificate signed by unknown authority
  • OneAgent unable to connect when using Istio
  • Connectivity issues when using Calico
  • Dynatrace Service could be missing when Istio is enabled
• Other
  • Monitoring not available problem for inactive Kubernetes cluster
  • K8s pod OneAgent missing code level monitoring when there is a delay to setup network
  • MountVolume Error while migrating from ClassicFullstack to CloudNativeFullStack monitoring Mode Dyna...
  • Troubleshooting PVC not showing data
  • [Kubernetes] Node memory usage is greater than memory limits
  • Why are there gaps in Kubernetes monitoring of a cluster after I change the Kubernetes settings for ...
  • How do I get logs from a pod that is crashing or in crashloopbackoff ? cloudNativeFullStack/PaaS/App...
  • ActiveGate/OneAgent pod won't start anymore. Failed to pull image on managed cluster.
  • Set a certain ActiveGate/OneAgent image version in the Dynatrace Operator
  • Container issues and initial information gathering
  • Which of my custom dashboards, custom alerts, and SLOs use deprecated Kubernetes metrics?
  • Solving Kubernetes Deployment Failures - CrashLoopBackOff
• Taints and Tolerations for Dynatrace resources
  • Dynatrace Operator - Having a different number of OA/CSI pods than actual K8S nodes
• Scraping Prometheus metrics
  • Excluded Prometheus metrics are still fetched by ActiveGate
  • How to troubleshoot missing Prometheus metrics
  • A warning message "Counter metric cache limit exceeded" in AG logs
• Problem detections
  • Troubleshooting Kubernetes 'High CPU Throttling' Alert Problems in Dynatrace
  • Troubleshooting Kubernetes CPU Throttling Problems in Dynatrace
• Generation of Operator's Support Archives:
  • Generate a support archive using the support-archive subcommand
• Security:
  • Kubernetes monitoring modes & security controls
• Injection
  • Dynatrace Operator - Cloud Native Full Stack - Pod Injection Validation and Troubleshooting

Known issues with Operator components

Below, you can find articles about known issues and their solutions:

Since	Solved
1.8.0	1.8.1	Operator 1.8.0 - OpenShift Operator Hub based install - API Monitoring missing
1.3.0	1.4.1	Kubernetes Operator CSI driver crashes frequently
1.1.0	1.5.1	OneAgent pods unable to validate SSL cert of containerized ActiveGate after upgrading to Dynatrace O...

 

How-to guides

• Configure monitoring for namespaces and pods (applies for Cloud Native Fullstack and Application-only deployments):
  • Monitor specific namespaces
  • Exclude specific namespaces
  • Exclude specific pods in monitored namespaces
  • Exclude specific containers in monitored pods
  • Monitor only specific pods
  • Fine-tuning of injection for applicationMonitoring without CSI driver
  • Dynatrace Pod Redeployment after image updates within private registry