topic Kubernetes Pod Phase Monitoring in Container platforms

Kubernetes Pod Phase Monitoring

sivart_89 — Thu, 14 Jul 2022 09:06:09 GMT

Using the Kubernetes: Pod counts metric (metric key builtin:cloud.kubernetes.workload.pod) you can filter by the pod phase which shows Running, Succeeded, Pending, or Failed. There are other states that a pod could be in such as Deleted (Terminated), see below. Am I missing something here? I specifically need to monitor for that state as well but can't seem to find a way.

Re: Kubernetes Pod Phase Monitoring

abhi — Thu, 14 Jul 2022 23:49:25 GMT

Hi @sivart_89,

Please post this request in our RFE forum as this should be an enchantment request.

Product Ideas posted here are reviewed by our Product Management, who will provide feedback on the suggestion and indicate if the RFE is accepted, planned, already implemented, not planned, etc.

https://community.dynatrace.com/t5/Dynatrace-product-ideas/idb-p/DynatraceProductIdeas

Re: Kubernetes Pod Phase Monitoring

florian_g — Mon, 18 Jul 2022 14:25:36 GMT

Hi @sivart_89 ,

please help me understand why you would like to count terminated pods? This sounds like a forever-increasing metric. What's the value of knowing how many pods were terminated (in the past)? Usually, if a pod is terminated, it's not there anymore. It's also no official pod phase in k8s. We just introduced this phase, as Dynatrace also shows you pods that are already gone/terminated.

best

florian

Re: Kubernetes Pod Phase Monitoring

sivart_89 — Fri, 29 Jul 2022 15:09:54 GMT

Hi Florian,

This particular app team has a cron job running in a pod and wanted to know when it has been terminated. I don't know too much about kubernetes but we ended up going with this metric instead (later found this after digging more) and so far that is what they want. Maybe it doesn't make sense to have that monitored, i don't know being I'm not proficient with kubernetes but do know that seems to be what they want.

[Deprecated] Kubernetes: Containers per workload

Re: Kubernetes Pod Phase Monitoring

florian_g — Mon, 01 Aug 2022 06:04:56 GMT

Hi @sivart_89,
thanks for the clarification. I guess your team wants to track successful vs. failed executions of their jobs. In Kubernetes, they can find this as part of the pod phase. In Dynatrace, this information is available in the "builtin:kubernetes.pods" (Kubernetes: Pod count) metric. Please note, that Kubernetes keeps a history of successful and failed pods from being deleted depending on what is defined in the cronjob (see successfulJobsHistoryLimit and failedJobsHistoryLimit here)

Best,
Florian