topic Kubernetes API server metrics in Container platforms

Kubernetes API server metrics

janne_olkoniemi — Mon, 26 Feb 2024 12:12:13 GMT

Is anyone in here pulling these kubernetes metrics on Dynatrace and we are especially interested about the Kubernetes API metrics which is quite a blind spot with our current monitoring setup with Dynatrace.

https://kubernetes.io/docs/reference/instrumentation/metrics/

I have tried to go trough the Dynatrace documentation regarding this one but haven't found any instructions on that side how to do this. This default prometheus support doesn't seem to work on this case since at least on our Rancher kubernetes environment the kubernetes API is running on containers but there is no pod for it.

https://docs.dynatrace.com/docs/shortlink/monitor-prometheus-metrics#annotate-prometheus-exporter-pods

Re: Kubernetes API server metrics

florian_g — Wed, 16 Oct 2024 06:59:40 GMT

Hi @janne_olkoniemi - would this solution work for you? https://www.youtube.com/watch?v=p4hUhK7plic

Best,

Florian

Re: Kubernetes API server metrics

janne_olkoniemi — Tue, 19 Nov 2024 09:49:34 GMT

Yes, I have working skeleton of OTEL configuration which pulls out kubernetes api & kubelet metrics currently.

The challenge we still have is that the Dynatrace support for prometheus histograms seems to still be a bit limited and many of the metrics we are after to are dropped by Dynatrace otpl interface.

Most of the metrics we are after are currently getting this kind of error and especially the metrics we are interested about are getting this kind of error:

Unsupported metric: 'kubelet_pod_start_duration_seconds' - Reason: UNSUPPORTED_METRIC_TYPE_CUMULATIVE_HISTOGRAM