Unable to install OneAgent in Kubernetes for cluster monitoring

[aades_kanna]

From the Deploy Dynatrace page, I'm following the steps as it is to install OneAgent in our Kubernetes cluster (bare metal, one master node vm and other worker node vm):

1. kubectl create namespace dynatrace
2. kubectl apply -f https://github.com/Dynatrace/dynatrace-operator/releases/download/v0.6.0/kubernetes.yaml
3. kubectl -n dynatrace wait pod --for=condition=ready --selector=app.kubernetes.io/name=dynatrace-operator,app.kubernetes.io/component=webhook --timeout=300s
4. kubectl apply -f dynakube.yaml

The first three gets done successfully but the while doing the last step, it gives this error.

"""

secret/kubernetes-monitor created
Error from server (InternalError): error when creating "dynakube.yaml": Internal error occurred: failed calling webhook "webhook.dynatrace.com": failed to call webhook: Post "https://dynatrace-webhook.dynatrace.svc:443/validate?timeout=2s": context deadline exceeded

"""

In the troubleshooting section from the official Dynatrace documentation, it's written it might be due to firewall rules (port 8443 should be allowed) but that was in GKE context. I've also allowed the port 8443 in our firewall. The issue still persists. Can someone please help out?

[techean]

Here the yaml components for deployment are trying to connect to endpoint on port 443. Could you please confirm the output for below from your server. Also make sure your are applying the correct file. Read if this configuration url in yaml resource needed to be replace with reference to documentation..

curl -kv https://dynatrace-webhook.dynatrace.svc:443/

[aades_kanna]

The curl command gives me output of

* Could not resolve host: dynatrace-webhook.dynatrace.svc; Unknown error
* Closing connection 0
curl: (6) Could not resolve host: dynatrace-webhook.dynatrace.svc; Unknown error

I've enabled port 443 on both master and worker node as well. Not sure why its not able to connect. Also can you please expand on your last sentence regarding replacing something? As far as i know im using all the files correctly

[prashanth_karre]

Hello Aades,

   In order to make it work you need to create an Ingress firewall rule on GKE Master Cluster for port 443. If you are using a shared VPC then you need to define there and if its local then define that firewall in your local project where the cluster is running.

Thanks

~Prashanth

[aades_kanna]

It's just a local project but I did allow port 443 rules in firewall (firewalld) and still i get the same error as before.

[techean]

Prashant below have commented that. But it should be clubbed with access on the server too through firewall policy updates. Getting an ingress controller rule configure will help to resolve the connectivity issue if you are using an ingress controller for request routes.

[aades_kanna]

But I am not using any ingress controller, I'm just stuck where this problem is coming from.

[techean]

I suspect this url is needed to be change in yaml resource where it have been defined. Kindly raise a support ticket so that the support team could take this up for better help.

[techean]

Can you confirm which kn8s flavor you are using form below

Dynatrace supports a variety of Kubernetes flavors. For some distributions we have limited support based on compatibility with upstream Kubernetes.

EKS - Amazon Elastic Kubernetes Service
AKS - Azure Kubernetes Service
GKE - Google Kubernetes Engine and Autopilot
Google Anthos
Red Hat OpenShift Container Platform
Red Hat OpenShift Dedicated
VMware Tanzu Kubernetes Grid Integrated Edition
Rancher
D2iQ Konvoy
Docker Enterprise
IBM Kubernetes Service
SUSE Container as a Service platform

[techean]

That that endpoint is being mentioned in some resource file should be replace by some working url. Kindly raise a support ticket if this is for any client.

[Sanket_Molavade]

Hi @aades_kanna 

have you got the solution to the above issue?

I am also facing the same issue can you please guide me?

Error Details:error when creating "CR.yaml": Internal error occurred: failed calling webhook "webhook.dynatrace.com": Post "https://dynatrace-webhook.dynatrace.svc:443/validate?timeout=2s"

Also while describe the pod getting below log,

[root@ip-10-20-96-197 Dynatrace]# kubectl logs -f dynatrace-webhook-74578d556-w7hjk -n dynatrace --previous
{"level":"info","ts":"2022-09-22T11:58:23.831Z","logger":"dynatrace-operator.version","msg":"Dynatrace Operator","version":"v0.3.0","gitCommit":"883392d56f46393692fdc5c7e6ee898985d50333","buildDate":"2021-11-25 13:40:40+00:00","goVersion":"go1.16.10","platform":"linux/amd64"}
I0922 11:58:24.882318 1 request.go:665] Waited for 1.035751229s due to client-side throttling, not priority and fairness, request: GET:https://172.20.0.1:443/apis/certificates.k8s.io/v1?timeout=32s
{"level":"info","ts":"2022-09-22T11:58:25.235Z","logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":":8383"}
{"level":"info","ts":"2022-09-22T11:58:25.235Z","msg":"SSL certificates configured","dir":"/tmp/webhook/certs","key":"tls.key","cert":"tls.crt"}
{"level":"info","ts":"2022-09-22T11:58:25.240Z","msg":"Waiting for certificate secret to be available."}
{"level":"info","ts":"2022-09-22T11:58:35.248Z","msg":"Failed to update certificates","error":"could not create cert directory: mkdir /tmp/webhook: read-only file system"}
{"level":"info","ts":"2022-09-22T11:58:45.257Z","msg":"Failed to update certificates","error":"could not create cert directory: mkdir /tmp/webhook: read-only file system"}

regards,

Sanket.

[saisandeep]

Hi @aades_kanna , @Sanket_Molavade, @techean 

have you solved the above problem. I am also facing the same problem. Can you please help me.

[manickam · Accepted solution]

Team before running this sudo kubectl apply -f dynakube.yaml make sure the path is correct where the yaml file is located

[Omkar164]

Hi Team,

Have you identified any solution for it I am also facing the same issue