on - edited on by
doweinberger
You can integrate your Prometheus exporter to Dynatrace and make them available for charting, alerting, and analysis.
If you completed the integration but still cannot find the relevant metrics on the Dynatrace metrics UI, then run the following steps for troubleshooting:
- Make sure the annotations are properly set (link)
- with pod template annotations
- in the Kubernetes app
- in the Kubernetes Classic app
- in the Kubernetes app
- with service annotations
- in the Kubernetes app
- in the Kubernetes Classic app
- in the Kubernetes app
- with pod template annotations
- “Monitor annotated Prometheus exporters” toggle is enabled from the Kubernetes integration settings
- Metric output format (text/plain) and exposed metric types are valid.
- Install “Kubernetes Monitoring Statistics” extension from the Dynatrace Hub. This extension provides you with self-monitoring statistics for all the Kubernetes clusters you already monitor within your Dynatrace environment and the root cause (response code) for failing API requests.
- Run one of the following curl commands valid for your deployment:
a) ActiveGate is running outside (non-containerized) the Kubernetes/OpenShift cluster
API=<enter external k8s api>
NAMESPACE=<enter pod namespace here e.g. openshift-node>
POD=<enter pod name here e.g. node-exporter-1234>
PORT=<enter port here e.g. 9100>
PATH=<enter path here e.g. /metrics>
TOKEN=<enter bearer token here>
curl https://$API/api/v1/namespaces/$NAMESPACE/pods/$POD:$PORT/proxy$PATH \
--connect-timeout 20 -v -k -H "Authorization: Bearer $TOKEN"
b) ActiveGate is running inside (containerized) the Kubernetes/OpenShift cluster
POD_OR_NODE_IP=<enter pod or node ip here>
METRICS_PORT=<enter port here e.g. 9100>
METRICS_PATH=<enter path here e.g. /metrics>
kubectl run -n dynatrace curl --image=curlimages/curl --restart=Never -it --rm \
-- http://$POD_OR_NODE_IP:$METRICS_PORT$METRICS_PATH \
--connect-timeout 20 -k -v
c) ActiveGate is running inside (containerized) the Kubernetes/OpenShift cluster and metrics endpoint requires RBAC authentication
POD_OR_NODE_IP=<enter pod or node ip here>
METRICS_PORT=<enter port here e.g. 9100>
METRICS_PATH=<enter path here e.g. /metrics>
TOKEN=<enter bearer token of dynatrace-kubernetes-monitoring service account here>
kubectl run -n dynatrace curl --image=curlimages/curl --restart=Never -it --rm \
-- http://$POD_OR_NODE_IP:$METRICS_PORT$METRICS_PATH \
--connect-timeout 20 -k -v -H "Authorization: Bearer $TOKEN"
- Use the latest available AG version since it is highly recommended.
- Collect AG logs and results from all the above steps and create a ticket for Dynatrace Technical Support for further investigation if the metrics are still missing.
