Kubernetes Pod Phase Monitoring

[sivart_89]

Using the Kubernetes: Pod counts metric (metric key builtin:cloud.kubernetes.workload.pod) you can filter by the pod phase which shows Running, Succeeded, Pending, or Failed. There are other states that a pod could be in such as Deleted (Terminated), see below. Am I missing something here? I specifically need to monitor for that state as well but can't seem to find a way.

[abhi]

Hi @sivart_89,

Please post this request in our RFE forum as this should be an enchantment request. 

Product Ideas posted here are reviewed by our Product Management, who will provide feedback on the suggestion and indicate if the RFE is accepted, planned, already implemented, not planned, etc.

https://community.dynatrace.com/t5/Dynatrace-product-ideas/idb-p/DynatraceProductIdeas

[florian_g]

Hi @sivart_89 ,

please help me understand why you would like to count terminated pods? This sounds like a forever-increasing metric. What's the value of knowing how many pods were terminated (in the past)? Usually, if a pod is terminated, it's not there anymore. It's also no official pod phase in k8s. We just introduced this phase, as Dynatrace also shows you pods that are already gone/terminated.

best

florian

[sivart_89]

Hi Florian,

This particular app team has a cron job running in a pod and wanted to know when it has been terminated. I don't know too much about kubernetes but we ended up going with this metric instead (later found this after digging more) and so far that is what they want. Maybe it doesn't make sense to have that monitored, i don't know being I'm not proficient with kubernetes but do know that seems to be what they want.

[Deprecated] Kubernetes: Containers per workload

[florian_g · Accepted solution]

Hi @sivart_89,
thanks for the clarification. I guess your team wants to track successful vs. failed executions of their jobs. In Kubernetes, they can find this as part of the pod phase. In Dynatrace, this information is available in the "builtin:kubernetes.pods" (Kubernetes: Pod count) metric. Please note, that Kubernetes keeps a history of successful and failed pods from being deleted depending on what is defined in the cronjob (see successfulJobsHistoryLimit and failedJobsHistoryLimit here)

Best,
Florian