cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Dynatrace managed intaller fails SSL: CERTIFICATE_VERIFY_FAILED

mirzet_kadic
Newcomer

Hi,

When I try to install dynatrace managed:

dynatrace-managed-1.191.0.20200316-134212.sh --tmp-dir "/var/opt/dynatrace-tmp" \
--install \
--install-silent \
--system-user "dynatrace:dynatrace" \
--license "XXXXXXXXXX" \
--initial-environment "main" \
--initial-first-name "Max" \
--initial-last-name "Musermann" \
--initial-email "max.mustermann@example.com" \
--initial-pass "Demo123!" \
--binaries-dir "/opt/dynatrace-managed" \
--network-proxy "http://server-proxy.example.ch:8080/" \
--datastore-dir "/var/opt/dynatrace-managed"

I'm getting the error;

Network proxy used for this installation in silent mode: http://server-proxy.example.ch:8080
Testing connection to Dynatrace Mission Control ... failed

/var/log/dynatrace/install.log says:

2020-03-25 13:41:34 UTC Network proxy used for this installation in silent mode: http://server-proxy.example.ch:8080
2020-03-25 13:41:34 UTC Testing connection to: https://mcsvc-dev.dynatracelabs.com:443, via proxy: http://server-proxy.example.ch:8080 ...
2020-03-25 13:41:35 UTC Testing connection ... failed, with error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)
2020-03-25 13:41:35 UTC Testing connection to: https://opcsvc-dev.ruxitlabs.com:443, via proxy: http://server-proxy.example.ch:8080 ...
2020-03-25 13:41:36 UTC Testing connection ... failed, with error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)
2020-03-25 13:41:36 UTC Cannot connect to Dynatrace Mission Control via network proxy http://server-proxy.example.ch:8080
2020-03-25 13:41:36 UTC Checking connection to Mission Control .. failed. Please check network and security settings

The proxy intercepts the SSL Traffic (MITM) using a certificate which is signed by a internal CA.

Can you help?

Thanks!

7 REPLIES 7

Radoslaw_Szulgo
Dynatrace Leader
Dynatrace Leader

See my instructions below. If you are installing the first node - follow steps 1-3. Then try with the installer. Let me know how it goes.

How to find the root cause of the issue?

See installer log to inspect issues in connectivity to Mission Control service. Installer log can be found at <DYNATRACE_BINARIES_DIR>/log/installer/*-install-of-managed-installer.log (for instance: 20190927-091357-success-install-of-managed-installer.log). Here’s a typical issue occurrence:


2019-09-27 07:18:58 UTC Testing connection tåo Dynatrace Mission Control https://mcsvc.dynatrace.com:443 ... failed, with error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)

Why installer cannot connect to Mission Control but the Server is running fine?

Installer checks Mission Control connectivity to configure the cluster to use either opcsvc.ruxit.com domain or mcsvc.dynatrace.com domain for all the communication endpoints. Installer use core OS certificate repository to validate the chain of SSL certificates exposed by Mission Control. Server use it’s keystore to check the chain.

If a proxy server or a firewall service is used between the cluster that intercepts the traffic, the verification of the chain also includes that proxy/firewall certificates to guarantee secure connection.

How to check if it works or not?

The simplest test is to use `curl` command. Example:

$ curl -x http://proxyuser:proxypass@outbound-proxy.dynatracelabs.com:8080 -I https://mcsvc.dynatrace.com


How to solve it?

In that situation it is required to update OS trusted root certificates. You need to prepare:

  • SSH access to the cluster node’s host
  • root permissions

Use the following steps to add Proxy/Firewall public certificate and chain to the system.

  1. Update OS trusted root certificates. Use hints below how to do it in a given Linux distribution:

Linux (Ubuntu)

Function

Method

Add

  1. Copy your CA to dir /usr/local/share/ca-certificates/
  2. Use command: sudo cp foo.crt /usr/local/share/ca-certificates/foo.crt
  3. Update the CA store: sudo update-ca-certificates

Linux (Red Hat / CentOs 6)

Function

Method

Add

  1. Install the ca-certificates package: yum install ca-certificates
  2. Enable the dynamic CA configuration feature: update-ca-trust force-enable
  3. Add it as a new file to /etc/pki/ca-trust/source/anchors/: cp foo.crt /etc/pki/ca-trust/source/anchors/
  4. Use command: update-ca-trust



2. Set system variable to configure OS trusted root certificates path:

export REQUESTS_CA_BUNDLE=<TRUSTED_CERT_REPO_DIR>

for instance for Red Hat:
export REQUESTS_CA_BUNDLE=/etc/pki/ca-trust/source/anchors/

3. For root user add that variable to bash profile, e.g. :

echo 'export REQUESTS_CA_BUNDLE=/etc/pki/ca-trust/source/anchors/' >> ~/.bash_profile


4. Run cluster node reconfiguration (note, it restarts all the services)

<DYNATRACE_BINARIES_DIR>/installer/reconfigure.sh

5. Check install.log and server config if Mission Control endpoint was updated successfully. Server config can be checked at:

<DYNATRACE_BINARIES_DIR>/server/conf/config.properties


And should contain:

#opc management url

opcUrl=https://mcsvc.dynatrace.com/


6. Repeat for all cluster nodes.

Technical Product Manager,
Dynatrace Managed expert

And that also might be needed to ensure later on Server has the connection as well:

https://www.dynatrace.com/support/help/setup-and-configuration/dynatrace-managed/configuration/how-t...

Technical Product Manager,
Dynatrace Managed expert

Thank you @Radoslaw S.


The hint with export REQUESTS_CA_BUNDLE solved the ssl problem but now i'm getting a new error:

2020-03-26 08:58:27 UTC Checking connection to Mission Control ..

2020-03-26 08:58:27 UTC Network proxy used for this installation in silent mode: http://server-proxy.example.ch:8080

2020-03-26 08:58:27 UTC Testing connection to: https://mcsvc-dev.dynatracelabs.com:443, via proxy: http://server-proxy.example.ch:8080 ...

2020-03-26 08:58:27 UTC Testing connection ... failed, with error: [Errno 21] Is a directory

2020-03-26 08:58:27 UTC Testing connection to: https://opcsvc-dev.ruxitlabs.com:443, via proxy: http://server-proxy.example.ch:8080 ...

2020-03-26 08:58:27 UTC Testing connection ... failed, with error: [Errno 21] Is a directory

2020-03-26 08:58:27 UTC Cannot connect to Dynatrace Mission Control via network proxy http://server-proxy.example.ch:8080

2020-03-26 08:58:27 UTC Checking connection to Mission Control .. failed. Please check network and security settings


the curl test is sucessfull:

[root@dyndemo dynatrace-managed-installer]# curl -x http://server-proxy.example.ch:8080 -I https://mcsvc.dynatrace.com

HTTP/1.1 200 Connection established

Proxy-Agent: Fortinet-Proxy/1.0

HTTP/1.1 200 OK

Server: nginx

Date: Thu, 26 Mar 2020 09:09:35 GMT

Content-Type: text/html; charset=utf-8

Content-Length: 78

Last-Modified: Mon, 23 Mar 2020 11:23:00 GMT

Connection: keep-alive

ETag: "5e789c14-4e"

Keep-Alive: timeout=55, max=16384

Accept-Ranges: bytes

my bad... REQUESTS_CA_BUNDLE should be set to the cert file instead of the directory. I'll update the procedure in my comment as well.

Let me know if that helped!

Technical Product Manager,
Dynatrace Managed expert

Tank you @Radoslaw S.

That solved the problem but the next one popped up:

2020-03-26 13:12:48 UTC REST GET ... url: https://mcsvc-dev.dynatracelabs.com:443/rest/public/v2.0/installation, user: installer ...

2020-03-26 13:12:49 UTC REST GET ... failed, with error 3 - invalid user name or password: {"code":401,"message":"It appears that you don't have permission to visit the page."}

2020-03-26 13:12:49 UTC Failed to get installation info from Mission Control - it can be caused by incorrect network proxy settings or by invalid license key

Installation failed, with status: system verified, connected to Mission Control after 1 minute 50 seconds.

Exit code: 3

Errors:

Failed to get installation info from Mission Control - it can be caused by incorrect network proxy settings or by invalid license key

2020-03-26 13:12:49 UTC Exit code is 3

===== End of installation, started at 2020-03-26 14:10:58, lasting 1m 50s =====


with curl I don't have any issues:

curl -x https://server-proxy.xaas.swissic.ch:8080 -I https://mcsvc-dev.dynatracelabs.com


the license is valid until Nov. 25, 2021

Installer was downloaded incorrectly. Additionally, license is already in use.

Technical Product Manager,
Dynatrace Managed expert

Thank you very much @Radoslaw S.

This issue is solved