"Deep monitoring is not activate" error message

[Nissou]

Hello everyone,

After I installed OneAgent with CloudNativeFullStack template in our AKS cluster, I'm not able to obtain any metrics. I have a message saying that the "Deep monitoring is not active" : 

Do you have any idea or did you face this issue before?

Best regards

[nanand3]

We are seeing this same issue within our environment, kindly let us know what could be the issue with cloudNative installation approach.

Our troubleshooting operator logs also shows error : "[31m X Unknown OneAgentCodeModules image[0m"

[gopher]

For GO apps, there are a variety of things that can prevent instrumentation.  
Have you checked your 'GO' rules in these 2 places within settings.

Monitored Technologies -> GO 

Also then check that the old GO static process monitoring rule is not enabled. There is a default duplicate inbuilt process monitoring rule that counters this primary enablement.

Process & containers -. built in monitoring rules.  -> rule 47

These will prevent static GO apps from being instrumented. 

[jo-ma]

We're seeing this as well in a cluster we're using to test the cloud native full stack deployment model.  Specifically we're seeing this issue with Go applications.  Java applications are being instrumented correctly.  We also have a single js frontend application that's being instrumented correctly, I noticed the OP screenshot is js.

[dannemca]

Can you all check for this? https://www.dynatrace.com/support/help/setup-and-configuration/setup-on-container-platforms/kubernet...

You may using some annotation/labels rules that are excluding the pods/namespaces.

[jo-ma]

In my case we're using a pretty vanilla dynakube config and are not explicitly adding any Dynatrace related annotations to our deployments / pods.  We're also not excluding any namespaces explicitly in the dynakube config.

We're seeing all Go services failing to be instrumented with deep monitoring, whether that's our services we've created in-house or third party type services like ingress-nginx or Prometheus.

From inspecting one of the pods for one of these Go services, it looks like oneagent is appending the correct annotations to indicate deep monitoring should be enabled:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    data-ingest.dynatrace.com/injected: "true"
    dynakube.dynatrace.com/injected: "true"
    oneagent.dynatrace.com/injected: "true"

[PedroDeodato]

We are facing this exact issue on multiple process, with multiple technologies (not only Go but Java - Springboot as well).

@Nissou , did you solve your issue? If so, can you share what you did, please? Thanks!

[anaidu]

Hi @PedroDeodato,

We are facing the same issue, was it resolved for you, if yes how and will taking a Process Restart resolve this? as someone from Support asked us to perform Process Restart. But as it is Production Env, we aren't 100% sure to test it with customer env.

[PedroDeodato]

Greetings, @anaidu ,

Unfortunately, I was never able to solve this issue: in our case, the Dynatrace Operator was not being able to instrument some processed because they were label as "managed by : AKS" (not the actual label, but it was something similar).

This was described as a blocker to the OneAgent instrumentation capabilities, and the Client was not willing to remove that label either (because it was key to several management features of Azure/AKS).

So, I gave up on Cloud Native Full Stack mode and uninstalled the Dynatrace Operator.

After that, I installed it again but using the "old" Classic Full Stack.

Bearing in mind that this mode requires process restarts, we asked the Client to do so and everything worked right away!

To this day, everything is working 100% fine with the Classic Full Stack mode, which is great!

I would suggest that you try the same: here's hoping that you achieve success! 🙂

[anaidu]

Hi @PedroDeodato ,

Thanks for sharing. Eventually we also went with Classic FS, as the ENV was Prod and we didn't have time to troubleshoot. Also for other guys opting Cloud Native, please be aware that if you're using nodeselector options for CSI drivers, it would even then inject binaries in other nodes that weren't selected for monitoring and it can fail those jobs like example cron jobs running. So either cover all the nodes or nothing. Anyways Classic Fullstack is working fine as of now, we need more accurate troubleshooting from Dynatrace Support/Documentation on Cloud Native as many guys are facing this same issue as we were.

Thanks,

Ajay

[javalosa]

I have same issue in Tomcat JAVA technology. This is a very serious problem. I need help 

[umeshdt2023]

I am also getting the same error for java, .net technologies in our cluster. Any suggestion on how to resolve this? Thanks in advance.