topic Re: Error while adding the new node to managed cluster in Dynatrace Managed Q&A

Error while adding the new node to Managed cluster

mayana_s_khan — Thu, 25 May 2023 09:35:49 GMT

Hi,

I am getting below error while adding the node to cluster. I am running the installer script as root user.

Installation failed, with status: installer unpacked, system verified, connected to Mission Control, connected to Dynatrace cluster, added to Dynatrace cluster, agent downloaded after 2 minutes 44 seconds.
Exit code: 5

Errors:
Installation failed, with error Dtrun doesn't work properly, check if command "/opt/dtrun/dtrun" is permitted to run with elevated privileges. The dtrun validation failed with error: sudo: PAM account management error: Authentication service cannot retrieve authentication info

Installation logs

2020-07-22 09:22:42 UTC old: RUNNER_PATH=''
2020-07-22 09:22:42 UTC new: RUNNER_PATH=/opt/dtrun/run
2020-07-22 09:22:42 UTC old: PRODUCT_USER=''
2020-07-22 09:22:42 UTC new: PRODUCT_USER=dynatrace
2020-07-22 09:22:42 UTC Setting files in path "/opt/dtrun/validate" as executable ...
2020-07-22 09:22:42 UTC Setting restricted permissions for path "/opt/dtrun/validate" ...
2020-07-22 09:22:42 UTC Preparing dtrun config ..
2020-07-22 09:22:42 UTC Saving config file: "/opt/dtrun/dtrun.conf"..
2020-07-22 09:22:42 UTC Setting restricted permissions for path "/opt/dtrun/dtrun.conf" ...
2020-07-22 09:22:42 UTC Saving list of commands in file .. done
2020-07-22 09:22:42 UTC Preparing dtrun config .. done
2020-07-22 09:22:42 UTC Preparing sudo rights .. skipped, different tool is used in system: $CMD
2020-07-22 09:22:42 UTC Installing dtrun tool .. done
2020-07-22 09:22:42 UTC Validating dtrun ..
2020-07-22 09:22:42 UTC Validating su privileges command "$CMD" ...
2020-07-22 09:22:42 UTC Validating su privileges command ... success
2020-07-22 09:22:42 UTC Executing command "/opt/dtrun/run" as user "dynatrace:dynatrace" ...
/opt/dtrun/run: line 20: /opt/dtrun/dtrun: Permission denied
2020-07-22 09:22:42 UTC Executing command as user ... failed with exitcode 126
2020-07-22 09:22:42 UTC Launching /opt/dtrun/run as child process failed
2020-07-22 09:22:42 UTC Executing command "/opt/dtrun/run" as user "dynatrace:dynatrace" ...
/opt/dtrun/run: line 20: /opt/dtrun/dtrun: Permission denied
2020-07-22 09:22:42 UTC Executing command as user ... failed with exitcode 126
2020-07-22 09:22:42 UTC Launching /opt/dtrun/run via runuser command failed
2020-07-22 09:22:42 UTC Validating dtrun .. failed
2020-07-22 09:22:42 UTC Installation failed, with error Dtrun doesn't work properly, check if command "/opt/dtrun/dtrun" is permitted to run with elevated privileges. The dtrun validation failed with error: /opt/dtrun/run: line 20: /opt/dtrun/dtrun: Permission denied

2020-07-22 09:22:42 UTC Removing this node from Dynatrace cluster ...

Can you please suggest how can i proceed further.

@Sebastian K.

Re: Error while adding the new node to managed cluster

Radoslaw_Szulgo — Wed, 22 Jul 2020 13:30:23 GMT

Some questions:

1. What system do you use?

2. Do you use Selinux policy ?

It looks like that the root cause here is that dynatrace user is not able to gain elevated privileges for command /opt/dtrun/dtrun

During Managed installation these privileges are automatically added to sudo config (if it's present and active in system) in following steps

Installer looks for the following line in /etc/sudoers file:
```
#includedir /etc/sudoers.d
```
If it's not present, it's added at the end of the file

Installer creates sudo config file /etc/sudoers.d/dynatrace with following contents:

Defaults:dynatrace !requiretty
Defaults:dynatrace !env_reset
dynatrace ALL=(root:root) NOPASSWD:/opt/dtrun/dtrun

These settings give the dynatrace user the ability to launch command /opt/dtrun/dtrun with root rights, without asking for password.

The privileges available to Managed user can be always validated via /opt/dtrun/validate command (it works also when launched by root) - it should return the following message: Dtrun validation succeeded.

Re: Error while adding the new node to managed cluster

Radoslaw_Szulgo — Wed, 25 Aug 2021 07:20:45 GMT

Just a blind guess: You might need to include in your privilege system /opt/dtrun/* files to resolve this.

Re: Error while adding the new node to managed cluster

mayana_s_khan — Wed, 22 Jul 2020 13:46:52 GMT

@Radoslaw S.,

We are using the Redhat Linux v7.6.

I can see the parameters which you mention is added to sudoers files.

[root@node mayana]# cat /etc/sudoers |grep /etc/sudoers.d
## Read drop-in files from /etc/sudoers.d (the # here does not mean a comment)
#includedir /etc/sudoers.d
#include /etc/sudoers.d/dynatrace
[root@node mayana]# cat /etc/sudoers.d/dynatrace |grep dyn
Defaults:dynatrace !requiretty
Defaults:dynatrace !env_reset
dynatrace ALL=(root:root) EXEC: ALL, NOPASSWD: /opt/dtrun/dtrun
[root@node mayana]#

Also the documentation link which you have added its not opening.

Re: Error while adding the new node to managed cluster

Radoslaw_Szulgo — Wed, 22 Jul 2020 14:25:52 GMT

This should help then: https://access.redhat.com/solutions/3679241 :
Please add sudo to the list of allowed services in your access control rules and make sure users do not have an expired password even in cases where ssh-keys are used for the login.

Root Cause
With sudo-1.8.23-1.el7 and later, the tool now evaluates the PAM account stack and therefore also enforces any account restrictions enforced through PAM. This affects any time and host based access control as well as password expiration.

Please also see the Red Hat Enterprise Linux 7.6 release notes:

sudo now runs PAM stack even when no authentication is required

With this update, the sudo utility runs Pluggable Authentication Module (PAM) account management modules even when the "NOPASSWD" option is configured in the policy. > This enables checking for restrictions imposed by PAM modules outside of the authentication phase. As a result, PAM modules, such as pam_time, now work properly in the described scenario.

PS sorry for the link - it's incorrect one.

Re: Error while adding the new node to managed cluster

mayana_s_khan — Wed, 22 Jul 2020 14:43:31 GMT

@Radoslaw S. Thanks for your help, I will ask linux admin to necessary changes. once the changes has been made i will give a try

Re: Error while adding the new node to managed cluster

mayana_s_khan — Thu, 27 Aug 2020 11:35:34 GMT

Actually we had two dynatrace users in existing cluster one dynatrace user in AD and second locally created dynatrace user which was causing issue in sudo.

We have removed the local dynatrace users from all cluster nodes and change the owner of associated directories to AD dynatrace user and restarted the cluster nodes. Then successfully added the new node to cluster