topic Re: Null values returned for pending pods in DQL

Null values returned for pending pods

VivekRagupathy — Tue, 18 Jul 2023 12:55:02 GMT

I am trying to filter the pending pods in one of the clusters and also only from any of the two namespaces, so I have written query like mentioned below,

I am getting null values as series, If there are no pending pods, I do expect that the query does not return any record.

Usually I take the last item from the array and consider that as the current state.

Am I missing anything here?

Query Used:

timeseries series=sum(dt.kubernetes.pods), by:{k8s.workload.name,k8s.namespace.name,k8s.cluster.name,pod_phase}
| filter k8s.cluster.name == "<myclustername>" and (startsWith(k8s.namespace.name, "<startofmynamespace1>") or contains(k8s.namespace.name, "<substringmynamespace2>")) and pod_phase == "Pending"

Result:

{
"series": [
null,
null,
null,
null,
null,
null,
null,
null,
null,
null,
null
],
"k8s.workload.name": "<myworkloadname>",
"k8s.namespace.name": "<mynamespace1>",
"k8s.cluster.name": "<Mycluster>",
"pod_phase": "Pending",
"timeframe": {
"start": "2023-07-18T10:19:00.000Z",
"end": "2023-07-18T10:30:00.000Z"
},
"interval": "60000000000"
}

Re: Null values returned for pending pods

sinisa_zubic — Tue, 18 Jul 2023 11:33:47 GMT

Hi @VivekRagupathy

I don't see in your query where you take the last item from the array and consider that as the current state?

Can you try out following query, it works for me in my environment

timeseries series=sum(dt.kubernetes.pods), by:{k8s.workload.name,k8s.namespace.name,k8s.cluster.name,pod_phase}, filter:pod_phase == "Pending" and k8s.workload.name == "" and k8s.namespace.name == "" and k8s.cluster.name == "" | fields countPendingPods = arrayLast(series), k8s.workload.name,k8s.namespace.name,k8s.cluster.name | filter isNotNull(countPendingPods)

with the arrayLast, I am retrieving the most recent value of the array.

Best,
Sini

Re: Null values returned for pending pods

VivekRagupathy — Tue, 18 Jul 2023 16:02:26 GMT

Hi Sini,
Thanks for looking in to this query.
Yes I removed the arrayLast() function to show all the null records returned.
Agree filtering with isNotNull() function is removing all the null records. Thanks for sharing this function.

I am curious to know when we are specifying pod_phase == "Pending" it should not return any null values right?

Re: Null values returned for pending pods

sinisa_zubic — Wed, 19 Jul 2023 10:40:39 GMT

for querying the dt.kubernetes.pods metric the pod_phase is a dimension, so you are filtering to that dimension. So the dimension with the value "Pending" of that metric is written usually once per minute only if a pod is in "Pending" state. If not, no it remains "null". that is the reason why the timeseries consists of null values (no pod was at that point in time in "Pending").

The result of the timeseries command is at least one record (depends on the dimension splitting you can specify). And a record consists of an array of datapoints which represent the value of a certain point in time.

I hope this explanation helps to understand you better what is happening?

Re: Null values returned for pending pods

VivekRagupathy — Wed, 19 Jul 2023 11:25:12 GMT

Hi Sini,

Thanks for the clarification. It helps 🙂