topic Re: Dynatrace OneAgent Pod Failing with CrashLoopBackOff in Kubernetes Cluster on Docker Desktop in Container platforms

Dynatrace OneAgent Pod Failing with CrashLoopBackOff in Kubernetes Cluster on Docker Desktop

Marwa — Mon, 19 Aug 2024 13:07:52 GMT

I am encountering an issue with the Dynatrace OneAgent pod in my Kubernetes cluster, which is provided by Docker Desktop. The pod dynatrace-montor-1-oneagent-jdpfs is in a CrashLoopBackOff state, while the dynatrace-oneagent-csi-driver-dx5pv pod is running without issues. The error logs indicate:

Error: Cannot determine volume host path from /proc/self/mountinfo: /data/kubelet/plugins/csi.oneagent.dynatrace.com/data/cfq65015/osagent does not exist on the host

Re: Dynatrace OneAgent Pod Failing with CrashLoopBackOff in Kubernetes Cluster on Docker Desktop

NavenduGupta — Sun, 06 Oct 2024 19:18:41 GMT

I have the exact same issue on windows 10.

Using minikube and docker on windows 10

19:10:24 Error: Cannot determine volume host path from /proc/self/mountinfo: /docker/volumes/minikube/_data/lib/kubelet/plugins/csi.oneagent.dynatrace.com/data/1cadc1b7-cc94-46a7-a5b8-47ade37d437f/osagent does not exist on the host

I notice that there is no response to this thread... any guidance?

Thanks in advance,

Navendu

Re: Dynatrace OneAgent Pod Failing with CrashLoopBackOff in Kubernetes Cluster on Docker Desktop

p_devulapalli — Mon, 07 Oct 2024 02:20:08 GMT

@Marwa @NavenduGupta , can you please check if you have enough space on the partition ?

also, do you see any error in events?

kubectl describe pod dynatrace-montor-1-oneagent-jdpfs -n dynatrace

Re: Dynatrace OneAgent Pod Failing with CrashLoopBackOff in Kubernetes Cluster on Docker Desktop

p_devulapalli — Mon, 07 Oct 2024 04:44:47 GMT

@NavenduGupta , are you using Helm or the manifest to setup? I've recently tried setting it up on minikube and I remember getting this error when using Helm . I would suggest using the manifest option as it worked for me and was bit easy to troubleshoot

Re: Dynatrace OneAgent Pod Failing with CrashLoopBackOff in Kubernetes Cluster on Docker Desktop

NavenduGupta — Mon, 07 Oct 2024 06:00:12 GMT

@p_devulapalli I'm using helm... will try that and get back.
I think the issue is with cloudNativeFullStack vs classicFullStack mode. When I downloaded the dynakuke.yaml, it default mode was set to cloudNativeFullStack and it was looking for the some image in /proc/self/mountinfo file which is not mounted/doesn't exist. In cloudNativeFullStack mode, probably we have to provide the image path in dynakube.yaml. May be minikube needs to be started with different options.
In classicFullStack mode, it worked.

Re: Dynatrace OneAgent Pod Failing with CrashLoopBackOff in Kubernetes Cluster on Docker Desktop

NavenduGupta — Mon, 07 Oct 2024 06:03:38 GMT

@p_devulapalli yes, there is enough space on the partition. As mentioned above, the issue is with cloudNativeFullStack vs classicFullStack mode. it works in classicFullStack mode and looks for an image at a specific location in cloudNativeFullStack mode.

Re: Dynatrace OneAgent Pod Failing with CrashLoopBackOff in Kubernetes Cluster on Docker Desktop

Peter_Youssef — Mon, 07 Oct 2024 06:23:14 GMT

Hello @Marwa , @NavenduGupta

Regarding the highlighted error simply increase or re-adjust the allocated oneagent resources.

By default, it's presented in Dynakube.yaml file

After increasing the allocated resources for example to 2x or 3x it should work as expected without issues.

KR,

Peter.

Re: Dynatrace OneAgent Pod Failing with CrashLoopBackOff in Kubernetes Cluster on Docker Desktop

NavenduGupta — Mon, 07 Oct 2024 08:07:03 GMT

@Peter_Youssef i noticed that the issue was because of classicFullStack vs cloudNativeFullStack mode. By default, in the dynakube.yaml file, OA was getting installed in cloudNativeFullStack mode. I changed it to classicFullStack model and it started working...

I'll try your solution in the cloudNativeFullStack mode and get back.

Regards,

Navendu

Re: Dynatrace OneAgent Pod Failing with CrashLoopBackOff in Kubernetes Cluster on Docker Desktop

Peter_Youssef — Mon, 07 Oct 2024 09:08:55 GMT

Thanks, keep us posted.
Hopefully it will work in cloudNativeFullStack mode.

It's recommended to proceed with K8s monitoring in cloudNativeFullStack whenever possible.

Best of Luck @NavenduGupta

Re: Dynatrace OneAgent Pod Failing with CrashLoopBackOff in Kubernetes Cluster on Docker Desktop

NavenduGupta — Mon, 07 Oct 2024 15:56:27 GMT

@Peter_Youssef @p_devulapalli slightly off topic here...

I'm running easytravel in my K8s cluster(minikube) and in classicFullstack mode,

1) when I connect it with SaaS, then I see all the kubernetes cluster/workloads/pods etc under the namespace I created in dynakube.yaml. I also see minikube top container (only one node for minikube with 8GB mem) as "host" under infrastructure. Here I have provided the "SaaS env url/api" for apiURL field
2) when I do the same with "managed", then I don't see top minikube under "host" under infrastructure. For managed, the apiUrl is "CAG url followed by env id /api". Here I get the kubernetes workload. but not the kubernetes top container under "host" under infrastructure... Also I'm not getting any easytravel application traffic under RUM. Even if my application detection may be wrong (i don't think so) but I should see the application traffic under "My web application" catch all application.

Only thing I see the difference between the AG in SaaS and Managed is that cluster AG doesn't have kubernetes module switched on/enabled. Could that be issue? but that doesn't make sense as the kubernetes AG is sending the data to cluster AG and it is only doing routing....

Any ideas?

Re: Dynatrace OneAgent Pod Failing with CrashLoopBackOff in Kubernetes Cluster on Docker Desktop

NavenduGupta — Wed, 09 Oct 2024 19:50:22 GMT

Anybody... any ideas?

Re: Dynatrace OneAgent Pod Failing with CrashLoopBackOff in Kubernetes Cluster on Docker Desktop

NavenduGupta — Wed, 09 Oct 2024 19:50:53 GMT

To All,

Any ideas?

Regards,

Navendu

Re: Dynatrace OneAgent Pod Failing with CrashLoopBackOff in Kubernetes Cluster on Docker Desktop

p_devulapalli — Wed, 09 Oct 2024 22:37:51 GMT

@NavenduGupta , can you please check if the below is enabled on your side?

Please ping me directly when you are free, we can take a look at it together if possible .

Re: Dynatrace OneAgent Pod Failing with CrashLoopBackOff in Kubernetes Cluster on Docker Desktop

p_devulapalli — Wed, 09 Oct 2024 22:51:47 GMT

@NavenduGupta other options you can try is to look at the operator logs

kubectl logs dynatrace-operator-xxxx -n dynatrace

Also try the troubleshoot command and see if you can find something

kubectl exec deploy/dynatrace-operator -n dynatrace -- dynatrace-operator troubleshoot

OneAgent logs

kubectl logs dynakube-oneagent-<id> -n dynatrace

Re: Dynatrace OneAgent Pod Failing with CrashLoopBackOff in Kubernetes Cluster on Docker Desktop

amosrodrigues1 — Sat, 31 May 2025 02:19:09 GMT

Hello @Marwa @NavenduGupta , I hope you and everyone are well 🙂

I have been following the discussions and I have also been facing problems provisioning the installation of the CloudNativeFullstack operator, so after several experiments I came up with an alternative that solved the problem for me and I hope it helps you.

Considering the current setup I use:

wsl2 with Ubuntu 24.04
Docker 28.2.1
Kubectl v1.33.1
Minikube v1.36.0

These were the adjustments I needed to make and that solved the problem for me:

# Check that exists sudo ls -l /var/lib/docker/volumes/k8s-dynatrace-labs/_data/lib/kubelet/plugins/csi.oneagent.dynatrace.com/data/_dynakubes/k8s-dynatrace-labs/ # Create if it does not exist sudo mkdir -p /var/lib/docker/volumes/k8s-dynatrace-labs/_data/lib/kubelet/plugins/csi.oneagent.dynatrace.com/data/_dynakubes/k8s-dynatrace-labs/osagent # Grants permission to directory sudo chmod -R 777 /var/lib/docker/volumes/k8s-dynatrace-labs/ # Minikube mounts volume on filesystem minikube mount /proc/self/mountinfo:/var/lib/docker/volumes/k8s-dynatrace-labs/_data/lib/kubelet/plugins/csi.oneagent.dynatrace.com/data/_dynakubes/k8s-dynatrace-labs/osagent -p k8s-dynatrace-labs Note: replace "/k8s-dynatrace-labs/" with the name of your minikube cluster profile added when creating "-p k8s-dynatrace-labs".

Best regards
Amós Rodrigues 🐧🚀