How to troubleshoot missing Prometheus metrics

shahinm — Mon, 13 Oct 2025 10:51:27 GMT

You can integrate your Prometheus exporter to Dynatrace and make them available for charting, alerting, and analysis.

If you completed the integration but still cannot find the relevant metrics on the Dynatrace metrics UI, then run the following steps for troubleshooting:

Make sure the annotations are properly set (link)
1. with pod template annotations
  1. in the Kubernetes app
  2. in the Kubernetes Classic app
2. with service annotations
  1. in the Kubernetes app
  2. in the Kubernetes Classic app
“Monitor annotated Prometheus exporters” toggle is enabled from the Kubernetes integration settings
Metric output format (text/plain) and exposed metric types are valid.
Install “Kubernetes Monitoring Statistics” extension from the Dynatrace Hub. This extension provides you with self-monitoring statistics for all the Kubernetes clusters you already monitor within your Dynatrace environment and the root cause (response code) for failing API requests.
Run one of the following curl commands valid for your deployment:

a) ActiveGate is running outside (non-containerized) the Kubernetes/OpenShift cluster

API=<enter external k8s api> NAMESPACE=<enter pod namespace here e.g. openshift-node> POD=<enter pod name here e.g. node-exporter-1234> PORT=<enter port here e.g. 9100> PATH=<enter path here e.g. /metrics> TOKEN=<enter bearer token here> curl https://$API/api/v1/namespaces/$NAMESPACE/pods/$POD:$PORT/proxy$PATH \ --connect-timeout 20 -v -k -H "Authorization: Bearer $TOKEN"

b) ActiveGate is running inside (containerized) the Kubernetes/OpenShift cluster

POD_OR_NODE_IP=<enter pod or node ip here> METRICS_PORT=<enter port here e.g. 9100> METRICS_PATH=<enter path here e.g. /metrics> kubectl run -n dynatrace curl --image=curlimages/curl --restart=Never -it --rm \ -- http://$POD_OR_NODE_IP:$METRICS_PORT$METRICS_PATH \ --connect-timeout 20 -k -v

c) ActiveGate is running inside (containerized) the Kubernetes/OpenShift cluster and metrics endpoint requires RBAC authentication

POD_OR_NODE_IP=<enter pod or node ip here> METRICS_PORT=<enter port here e.g. 9100> METRICS_PATH=<enter path here e.g. /metrics> TOKEN=<enter bearer token of dynatrace-kubernetes-monitoring service account here> kubectl run -n dynatrace curl --image=curlimages/curl --restart=Never -it --rm \ -- http://$POD_OR_NODE_IP:$METRICS_PORT$METRICS_PATH \ --connect-timeout 20 -k -v -H "Authorization: Bearer $TOKEN"

Review the output of curl, especially content-type header as it must be equal text/plain, which corresponds to Prometheus text format.
Use the latest available AG version since it is highly recommended.
Collect AG logs and results from all the above steps and create a ticket for Dynatrace Technical Support for further investigation if the metrics are still missing.

Re: How to troubleshoot missing Prometheus metrics

ChadTurner — Fri, 12 May 2023 18:56:42 GMT

Thank you for sharing this! I'm sure it will help out many community members. Always check your AG Version if you have issues lol

article How to troubleshoot missing Prometheus metrics in Troubleshooting

How to troubleshoot missing Prometheus metrics

Re: How to troubleshoot missing Prometheus metrics