topic DQL for filtering kubernetes pods by annotations in DQL

DQL for filtering kubernetes pods by annotations

roberto_camp1 — Tue, 16 Jul 2024 13:32:14 GMT

I am trying to use DQL to filter workloads in my EKS environment by certain pod annotations.

So far I have this:

fetch dt.entity.cloud_application_instance, from: -30m | fields id, pod.id = id, pod.name = entity.name, pod.containers = record(running = runningContainersCount, desired = desiredContainersCount),pod.condition = currentCondition, pod.labels = cloudApplicationLabels, pod.annotations = kubernetesAnnotations,pod.age = record(start = resourceCreationTimestamp, end = if(isNull(resourceDeletionTimestamp), now(), else:resourceDeletionTimestamp)), pod.resourceDeletionTimestamp = resourceDeletionTimestamp, pod.phase = cloudApplicationInstancePhase, cluster.id = clustered_by[dt.entity.kubernetes_cluster], node.id = runs_on[dt.entity.kubernetes_node],namespace.id = belongs_to[dt.entity.cloud_application_namespace], workload.id = instance_of[dt.entity.cloud_application], namespace.name = namespaceName, workload.name = workloadName| fieldsAdd pod.state = if((isNotNull(pod.resourceDeletionTimestamp) AND (pod.phase == "RUNNING")), "TERMINATING", else:pod.phase)
// Filters: [cluster,namespace]
| filter in(id, classicEntitySelector("type(CLOUD_APPLICATION_INSTANCE),toRelationship.isClusterOfCai(type(KUBERNETES_CLUSTER),entityName.equals(operations))"))
| filter in(id, classicEntitySelector("type(CLOUD_APPLICATION_INSTANCE),toRelationship.isNamespaceOfCai(type(CLOUD_APPLICATION_NAMESPACE),entityName.equals(client-services))"))
| filter in(id, classicEntitySelector("type(CLOUD_APPLICATION_INSTANCE),toRelationship.isClusterOfCai(type(KUBERNETES_CLUSTER),appEnabled(true))"))
| limit 200

What I am trying to do next is further filter whether or not each pod has certain annotations on it. For example lets' say the annotation is "instrumentation.opentelemetry.io/inject-javamonitoring/universal-instrumentation", but in reality , I just want to find any pods that have annotations starting with "instrumentation.opentelemetry.io".

How can add that type of filter to my existing DQL?

Re: DQL for filtering kubernetes pods by annotations

krzysztof_hoja — Tue, 16 Jul 2024 15:55:23 GMT

Actually checking specific annotation is easier than pattern. To do this use such filter and pod.annotations fileds already extracted by the query:

| filter isNotNull(pod.annotations[`instrumentation.opentelemetry.io/inject-javamonitoring/universal-instrumentation`])

backticks need to be used, because this annotation contains /

Checking patterns is not as straight forward, because annotations are complex record and we do not have means to analyze records in generic way. To do so, for now, we need to convert complex record into array of key-value pairs and use iterative expressions to look into it.

| fieldsAdd pod.annotations.array=parse(toString(pod.annotations), """'{' ARRAY{SPACE? '"' [^"]+:key '"' ':' JSON_VALUE:value ','?}*:a """) | filter iAny( startsWith(pod.annotations.array[][key], "instrumentation.opentelemetry.io" ))

I hope it helps

Re: DQL for filtering kubernetes pods by annotations

roberto_camp1 — Tue, 16 Jul 2024 16:22:45 GMT

Thank you krzysztof_hoja , yes it's a bit messy but the second example you give fits the solution perfectly. Thanks so much for a quick response!

PREVIEW