This website uses Cookies. Click Accept to agree to our website's cookie use as described in our Privacy Policy. Click Preferences to customize your cookie settings.
Accept
Reject
Preferences
FAQ
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Watchdog causing issues with Oneagent in cloudNativeFullStack deployment.

chahat_bhatia
Newcomer

‎05 Jun 2024 01:01 AM - last edited on ‎05 Jun 2024 07:28 AM by Community Team

I recently installed the Cloud Native Full Stack  for dynatrace. 
All the components seem to come up fine except `eks-sit1b-oneagent`.

 

NAME                                                         READY  STATUS 
dynatrace-oneagent-csi-driver-gwphr       4/4 Running
dynatrace-oneagent-csi-driver-r2bk9         4/4 Running 
dynatrace-operator-6c44b8fcff-2zhv6       1/1 Running 
dynatrace-webhook-5cb47fd4cd-pdx85     1/1 Running
dynatrace-webhook-5cb47fd4cd-qlhnj       1/1 Running
eks-sit1b-activegate-0                                  1/1 Running
eks-sit1b-oneagent-bhhqt                            0/1 Running 
eks-sit1b-oneagent-hpnv7                            0/1 Running 


The oneagent pod won't come up; logs in the pod:
dynatrace-oneagent Cannot open log file "/var/log/dynatrace/oneagent/watchdog/oneagentwatchdog_218530.log": error code: 13 (Permission denied) 
dynatrace-oneagent Cannot open log file "/var/log/dynatrace/oneagent/watchdog/oneagentwatchdog_218530.log": error code: 13 (Permission denied) 
dynatrace-oneagent Cannot open log file "/var/log/dynatrace/oneagent/watchdog/oneagentwatchdog_218530.log": error code: 13 (Permission denied) 
dynatrace-oneagent Cannot open log file "/var/log/dynatrace/oneagent/watchdog/oneagentwatchdog_218530.log": error code: 13 (Permission denied) 
dynatrace-oneagent Cannot open log file "/var/log/dynatrace/oneagent/watchdog/oneagentwatchdog_218530.log": error code: 13 (Permission denied)


The readiness probe failing with:

[2024-06-04 23:43:54.085] [error] Watchdog pid file not found
[2024-06-04 23:43:54.085] [info] Watchdog healthcheck returned FAILURE


Also some other pods in the cluster, erroring out with logs:
>>  MountVolume.SetUp failed for volume "oneagent-bin" : kubernetes.io/csi: mounter.SetUpAt failed to get CSI client: driver name csi.oneagent.dynatrace.com not found in the list of registered CSI drivers

But the CSI Driver is very much present on the cluster 😕


Any ideas on how to fix these issues? Even the slighest help would be appreciated. Thanks.

Labels:
Reply
4 REPLIES 4

Esam_Eid
Pro

‎05 Jun 2024 05:23 AM

Have you installed it by root user?

0 Kudos
Reply

PacoPorro
Dynatrace Leader
Dynatrace Leader

‎05 Jun 2024 07:44 AM

Can you check if something is blocking the watchdog ports?
On startup, OneAgent watchdog tries to open the first available port between ports 50000 and 50100.

0 Kudos
Reply

chahat_bhatia
Newcomer

‎05 Jun 2024 12:56 AM - last edited on ‎05 Jun 2024 07:24 AM by Community Team

I recently installed the Cloud Native Full Stack  for dynatrace. 
All the components seem to come up fine except `eks-sit1b-oneagent`.

 

NAME                                                         READY  STATUS 
dynatrace-oneagent-csi-driver-gwphr       4/4 Running
dynatrace-oneagent-csi-driver-r2bk9         4/4 Running 
dynatrace-operator-6c44b8fcff-2zhv6       1/1 Running 
dynatrace-webhook-5cb47fd4cd-pdx85     1/1 Running
dynatrace-webhook-5cb47fd4cd-qlhnj       1/1 Running
eks-sit1b-activegate-0                                  1/1 Running
eks-sit1b-oneagent-bhhqt                            0/1 Running 
eks-sit1b-oneagent-hpnv7                            0/1 Running 


The oneagent pod won't come up; logs in the pod:
dynatrace-oneagent Cannot open log file "/var/log/dynatrace/oneagent/watchdog/oneagentwatchdog_218530.log": error code: 13 (Permission denied) 
dynatrace-oneagent Cannot open log file "/var/log/dynatrace/oneagent/watchdog/oneagentwatchdog_218530.log": error code: 13 (Permission denied) 
dynatrace-oneagent Cannot open log file "/var/log/dynatrace/oneagent/watchdog/oneagentwatchdog_218530.log": error code: 13 (Permission denied) 
dynatrace-oneagent Cannot open log file "/var/log/dynatrace/oneagent/watchdog/oneagentwatchdog_218530.log": error code: 13 (Permission denied) 
dynatrace-oneagent Cannot open log file "/var/log/dynatrace/oneagent/watchdog/oneagentwatchdog_218530.log": error code: 13 (Permission denied)

The readiness probe failing with:

[2024-06-04 23:43:54.085] [error] Watchdog pid file not found
[2024-06-04 23:43:54.085] [info] Watchdog healthcheck returned FAILURE

Also some other pods in the cluster, erroring out with logs:
>>  MountVolume.SetUp failed for volume "oneagent-bin" : kubernetes.io/csi: mounter.SetUpAt failed to get CSI client: driver name csi.oneagent.dynatrace.com not found in the list of registered CSI drivers

But the CSI Driver is very much present on the cluster 😕


Any ideas on how to fix these issues? Even the slighest help would be appreciated. Thanks.

Reply
Ask a question

Featured Posts