topic Re: How to filter built-in CPU usage metric by cluster & namespace (and get max per workload)? in Container platforms

How to filter built-in CPU usage metric by cluster and namespace and get max per workload?

Aboud1 — Wed, 17 Dec 2025 11:37:22 GMT

Hi Community,

I’m using the built-in metric query in Dynatrace for CPU usage per workload, for example:

builtin:containers.cpu.usageMilliCores:filter(in("dt.entity.container_group_instance", entitySelector("type(CONTAINER_GROUP_INSTANCE), fromRelationships.IS_CGI_OF_CA(type(CLOUD_APPLICATION), entityId(CLOUD_APPLICATION-XXXXX))"))):parents:parents:splitBy("dt.entity.cloud_application"):sum

👉 This built-in [used by dynatrace by default] works perfectly when I analyze a single workload.

What I want to achieve:

Get the maximum CPU usage (peak value) per workload
For all workloads inside a specific cluster (OpenShift Main-Cluster) and a specific namespace (dxl-vf-consumer)
Without having to check each workload individually.

My challenge:
I’m not sure how to add the cluster + namespace filters into this built-in query.

Question:
How can I adjust this built-in query to include cluster and namespace filters so that it returns the max CPU usage per workload within that scope?

Re: How to filter built-in CPU usage metric by cluster & namespace (and get max per workload)?

p_devulapalli — Fri, 05 Sep 2025 00:26:55 GMT

@Aboud1 Can you please try the below and see if it helps ?

builtin:kubernetes.workload.cpu_usage:filter(and(or(in("dt.entity.cloud_application_namespace",entitySelector("type(cloud_application_namespace),entityName.equals(~"namespace_abc~")"))),or(in("dt.entity.kubernetes_cluster",entitySelector("type(kubernetes_cluster),entityName.equals(~"cluster_xyz~")"))))):splitBy("k8s.workload.name"):max:sort(value(max,descending))

Re: How to filter built-in CPU usage metric by cluster & namespace (and get max per workload)?

Aboud1 — Wed, 10 Sep 2025 06:01:42 GMT

Hello @p_devulapalli

Thanks for your reply 🙏

I tried the query you suggested with builtin:kubernetes.workload.cpu_usage and it works fine in terms of filtering by cluster and namespace.

But what I noticed is that the values are quite different from the ones I get when using builtin:containers.cpu.usageMilliCores (the metric I shared in my original question).

Especially during spikes, the difference becomes very large.
For example:

At 2025-09-03 02:17, using containers.cpu.usageMilliCores, the CPU usage for a workload was 1.29 cores.
At the exact same time, with kubernetes.workload.cpu_usage, it showed only 496 millicores.

When there’s no spike, the values are close, but during peaks the workload metric seems much lower.

So my question is:
👉 Why does this difference happen? Is the workload metric doing some kind of aggregation that smooths out spikes? If yes, does that mean it’s less accurate for identifying peak usage per workload?

Re: How to filter built-in CPU usage metric by cluster & namespace (and get max per workload)?

p_devulapalli — Wed, 10 Sep 2025 09:46:18 GMT

@Aboud1 There is a bit of difference between both the queries

builtin:containers.cpu.usageMilliCores: This metric represents CPU usage at the container level, measured in millicores. It provides detailed insights into individual container performance.

builtin:kubernetes.workload.cpu_usage: This metric aggregates CPU usage at the workload level, which includes all containers within a specific Kubernetes workload (e.g., Deployment, StatefulSet). Ideally this might include more than one container/pod as per the workload specifications

Since you mentioned about reporting on "max CPU usage per workload" I've suggested using builtin:kubernetes.workload.cpu_usage as it aggregates CPU usage at the workload level and also allows us to split the data by dimensions we are after unlike builtin:containers.cpu.usageMilliCores which do not provide those dimensions

Alternatively, if you are on SaaS you can extract the container level CPU using the below DQL with the needed dimensions

Hope this helps

timeseries max(dt.kubernetes.container.cpu_usage), by: { k8s.container.name, k8s.pod.name }, filter: { matchesValue(entityAttr(dt.entity.kubernetes_cluster, "entity.name"), "cluster_xyz") AND matchesValue(entityAttr(dt.entity.cloud_application_namespace, "entity.name"), "namespace_abc") }

Re: How to filter built-in CPU usage metric by cluster & namespace (and get max per workload)?

Aboud1 — Tue, 16 Sep 2025 05:12:53 GMT

Thanks a lot @p_devulapalli for the detailed clarification 🙏

Now it’s much clearer why the values differ between:

builtin:containers.cpu.usageMilliCores → container-level, more accurate for spikes
builtin:kubernetes.workload.cpu_usage → workload-level, aggregated across containers, so spikes appear lower

My main objective is still to get the peak CPU usage per workload in a given cluster + namespace.

As you mentioned, DQL would indeed solve this by combining container-level accuracy with the right filters. Unfortunately, I’m on Dynatrace Managed, so I don’t have DQL available — and that’s where the limitation bites me.

So basically, I need to decide between:

Accuracy (container-level) but no cluster/namespace filters, or
Scope (workload-level with filters) but with aggregated values.

That explains the behavior I was seeing. Thanks again for the guidance 🙌