ActiveGate plugin can't access CA Certificates (Python certificate)

r_weber — Thu, 18 May 2023 13:01:46 GMT

For those who might encounter this as well when writing their AG Remote Plugins a little hint.

While building another plugin with dependencies to another library I was confronted with an error that the included urllib3 library of the remoteplugin environment couldn't load RooT CA certificates. The error in the plugin's log looked something like this:

ERROR   [Python][15052339058944823227][Hostunit Consumption][140518812161792][ThreadPoolExecutor-0_2] - [set_full_status] (0)
Reason: SSLError
[Errno 2] No such file or directory
Traceback (most recent call last):
  File "/opt/dynatrace/remotepluginmodule/agent/plugin/engine.zip/site-packages/urllib3/util/ssl_.py", line 319, in ssl_wrap_socket
    context.load_verify_locations(ca_certs, ca_cert_dir)
FileNotFoundError: [Errno 2] No such file or directory

After digging into it I found that the invoked external SSL connection required the root certificates that are typically shipped with urllib3 or certifi package. (both are included in the remoteplugin runtime environment). However the virtual (?) path would try to find the CA certificate package at this path:

/opt/dynatrace/remotepluginmodule/agent/plugin/engine.zip/site-packages/certifi/cacert.pem

This is not a real path but points to the content of the site.zip archive. Python certifi gets this path by calling certifi.where() and urllib then fails to get the file, leading to this error.

To resolve this issue I had to explicitly point my plugin to an "real" cacert.pem file in the AG's filesystem that is not within a zip file. I manually added ca cacert.pem file (which could be enahanced with corporate CA certs as well to my plugin directory:

/opt/dynatrace/remotepluginmodule/plugin_deployment/custom.remote.python.missingmetrics/cacert.pem

Just posting this here because I was pretty surprised to see this. Maybe it's a bug in the plugin runtime as well that can be fixed permanently as well...

Re: Active Gate Plugin can't access CA Certificates (python certifi)

Mike_L — Fri, 29 Jan 2021 09:21:42 GMT

This is also how we've resolved this with extensions our services team has created for customers. I never dug into it as deep as you did with the root cause though. Thanks for sharing!

Re: Active Gate Plugin can't access CA Certificates (python certifi)

ChadTurner — Wed, 17 Feb 2021 23:45:15 GMT

@Reinhard W. thanks for posting this!

topic Re: Active Gate Plugin can't access CA Certificates (python certifi) in Extensions

ActiveGate plugin can't access CA Certificates (Python certificate)

Re: Active Gate Plugin can't access CA Certificates (python certifi)

Re: Active Gate Plugin can't access CA Certificates (python certifi)