Kubernetes application only monitoring pods availability

[danielD]

Hello,

this question is specifically for kubernetes app-only deployments from the operator. 

There are times when we see a pod that should have dynatrace enabled on it, and it detects a container, but it doesn't show the container being monitored in Dynatrace. Is there a way that we can alert when there's a pod where we should be seeing metrics / the one agent should be instrumented but for some reason it's not?

As an example: 

This pod says that it has a container in dynatrace

but when I go to the container group the last container dynatrace saw was from the previous pod from yesterday.

when we check on the pod it looked like it should have the one agent on the container correctly.

Could anyone help troubleshoot? 

thanks

[ChadTurner · Accepted solution]

Dynatrace does allow you to alert on missing data and this is a great use case for that feature. You will do this via Metric events that can be found under the UI Settings>Anomaly Detection>Metric Events. AS you target a value say CPU Usage there will be a section to set the alert threshold and there will also be an option to alert on missing data. That's what you will want as the data would be missing and an alert would be fired. 

Perform 2023 is right around the corner so maybe there will be an announcement on Kubernetes Monitoring enhancements pertaining to alerting. Sign up for Perform Virtual if you haven't done so already, its free!