Tagging Kubernetes Clusters, Workloads, and Pods

[shakib]

I’m needing to tag all entities (workloads, pods, & nodes) in a Kubernetes cluster via auto-tagging rules.

There is no way for me to do this with the rule type as “Monitored Entity” as a Kubernetes entity type (workloads, nodes, or anything K8 related) are not available to select. So, I assume the only way I can do this is by choosing the Entity Selector option when creating a tagging rule. But I need a way to tag 100+ workloads and 200+ pods without creating 300+ rules.

I was hoping I could tag all entities under a Management Zone but Dynatrace doesn’t allow that as specifying a Management Zone yields an error. I then thought I could do this at the cluster level but adding a tag to the cluster doesn’t cause it to pass down <type(KUBERNETES_CLUSTER),entityName("myK8ClusterName")>. According to Dynatrace entity types here are the seemingly relevant properties I need:

• Kubernetes Pods = type(CLOUD_APPLICATION_INSTANCE)
• Kubernete workload = type(CLOUD_APPLICATION)
• Kubernete Cluster = type(KUBERNETES_CLUSTER).

It looks like I need to string together these values in order to be able to tag what I need but I am running into issues getting it to work. I talked to Dynatrace support via chat and I was given an example of what worked for them before (but doesn’t work for me because it relies in a pre-existing tag).

• type(CONTAINER_GROUP_INSTANCE),toRelationships.isPgiOfCgi(types(PROCESS_GROUP_INSTANCE),tag([Kubernetes]app.kubernetes.io/instance:dev-atlas))

That support agent couldn’t get me a working syntax for what I need for the tagging rule despite the above example. But I’m looking do something like:

“Tag all pods where type is KUBERNETES_CLUSTER and cluster name is ShakibsClusterName"

Is this even possible via automated tagging rules? 

[ChadTurner · Accepted solution]

From an automated dynamic tagging stand point it isn't. But if you are willing to do a 1 to 1 it can be done via relations. For example. lets say I have a Namespace that resides under a cluster and I want to tag it with my own value. You can use the entity selector and leverage relationship tagging. via this string: 

Apply '<EASYTRAVELCLUSTER>' to Entities matching type("Cloud_Application_Namespace"), toRelationships.isClusterOfNamespace(type("KUBERNETES_CLUSTER"), entityName("<CLUSTERNAME/ID>"))

Now it will apply that "Easytravel" as my custom tag value to the namespace if and when that namespace is related to the defined cluster via Dynatrace AI. You can then do this for the other aspects such as Pods. 

It requires some thinking out of the box and isn't truly scalable for large organizations as you have to define them one by one. 

[ChadTurner]

@shakib I also have created a RFE that is requesting the ability to leverage pod labels as tags and such. this would help reduce the usage of the entity selector. Dynatrace needs to expand the dynamic tagging capabilities to all the auto discovered properties. Feel free to give it a like 🙂 

https://community.dynatrace.com/t5/Dynatrace-product-ideas/Expand-Dynamic-Auto-Tag-constructs-for-Ku... 

[shakib]

Thank you, Chad, that worked perfectly for the name spaces.  

If I have 10 name spaces then it seems I'll have to create 10 individual rules to cover each workload inside each individual namespace. 

I was able to use the same logic for Workloads and the working syntax below shows all workloads being tagged for namespace XYZ which belongs to cluster ABC.

• type(CLOUD_APPLICATION),toRelationships.isNamespaceOfCa(type("CLOUD_APPLICATION_NAMESPACE"), entityName("XYZ"), toRelationships.isClusterOfNamespace(type("KUBERNETES_CLUSTER"), entityName("ABC")))

I'm going to try the same approach for pods next and see how that works out. 

It's far from ideal to have to create this many rules. I really hope Dynatrace addresses this for future users so they don't have to suffer. 

[BaliBan]

hey @shakib ,

can you provide logical syntax  for pods and nodes also like you provide for workload

[DanielS · Accepted solution]

I share my collection of rules for Kubernetes Cluster:

• type(KUBERNETES_CLUSTER),entityName.in("<YOUR KUBERNETES CLUSTER NAME>")
• type(CLOUD_APPLICATION_NAMESPACE),toRelationships.isClusterOfNamespace(type(KUBERNETES_CLUSTER),entityName.in("<YOUR KUBERNETES CLUSTER NAME>"))
• type(CLOUD_APPLICATION),toRelationships.isClusterOfCa(type(KUBERNETES_CLUSTER),entityName.in("<YOUR KUBERNETES CLUSTER NAME>"))
• type(KUBERNETES_SERVICE),toRelationships.isClusterOfKubernetesSvc(type(KUBERNETES_CLUSTER),entityName.in("<YOUR KUBERNETES CLUSTER NAME>"))
• type(CONTAINER_GROUP),toRelationships.isInstanceOf(type(CONTAINER_GROUP_INSTANCE),fromRelationships.isCgiOfCluster(type(KUBERNETES_CLUSTER),entityName.in("<YOUR KUBERNETES CLUSTER NAME>")))
• type(CONTAINER_GROUP_INSTANCE),fromRelationships.isCgiOfCluster(type(KUBERNETES_CLUSTER),entityName.in("<YOUR KUBERNETES CLUSTER NAME>"))
• type(HOST),toRelationships.isClusterOfHost(type(KUBERNETES_CLUSTER),entityName.in("<YOUR KUBERNETES CLUSTER NAME>"))
• type(PROCESS_GROUP),fromRelationships.runsOn(type(HOST),toRelationships.isClusterOfHost(type(KUBERNETES_CLUSTER),entityName.in("<YOUR KUBERNETES CLUSTER NAME>")))
• type(PROCESS_GROUP_INSTANCE),fromRelationships.isProcessOf(type(HOST),toRelationships.isClusterOfHost(type(KUBERNETES_CLUSTER),entityName.in("<YOUR KUBERNETES CLUSTER NAME>")))
• type(SERVICE),fromRelationships.runsOnHost(type(HOST),toRelationships.isClusterOfHost(type(KUBERNETES_CLUSTER),entityName.in("<YOUR KUBERNETES CLUSTER NAME>")))

Only Manual Tags:

• type(CLOUD_APPLICATION_INSTANCE),toRelationships.isClusterOfCai(type(KUBERNETES_CLUSTER),entityName.in("<YOUR KUBERNETES CLUSTER NAME>"))

[BaliBan]

I have tried your first logic , but it does't showing any output even through i got entity matching ?

• type(KUBERNETES_CLUSTER),entityName.in("<YOUR KUBERNETES CLUSTER NAME>")
• type(KUBERNETES_CLUSTER),entityName.in("minikube")

hope can you answer my query

[DanielS]

Hi @BaliBan if you mean through the UI there is no place where you can see the output, but for example if you add to a MZ or reference the Kubernetes Cluster the tags are there. If you use the API you can see the tag reference: