Show Container name on Kubernetes container restart alerts/problem cards

[gurugogi]

Hi All,

I have enabled alerting for container restart at anomaly detection-> Workload, it is creating alerts/Problem card when any container gets restarted,  it is printing workload name in alerts but I want the particular container name to be printed in alerts (container name which got restarted.).

I tried with custom alerts(metric events), auto-tags etc looks like its not working, please guide if anyone has implemented this by modifying message body in mail alerts/slack alerts etc.

Thanks,
Guru Gogi 

[gurugogi]

anyone tried ?

[Mizső]

Hi @gurugogi,

Have you tride this metric experession in the metric event?

Then you can use the container dimension in the metric event:

I hope it helps.

Best regards,

Mizső

[gurugogi]

@Mizső  thanks for reply.
I tried below expression and its triggering alerts for container restarts.
builtin:kubernetes.container.restarts
:splitBy("k8s.container.name","k8s.pod.name","k8s.cluster.name","k8s.namespace.name")
:count:sort(value(avg,descending))

I have not sleeted any MZ for this metric event so it is creating Problems in default alerting profile but I want the Problems to be grouped under specific alerting profiles tagged to specific 

[gurugogi]

I want to use this single "Metric event" for all k8s clusters alerting, particularly we use slack notifications, so I want some particular container restart should be grouped under that particular alerting profile.

[gurugogi]

Only option is to create "Metric event"/"Custom alert" for each K8S cluster and MZ filter ?

[Peter_Youssef]

Hello @gurugogi 

1. let's agree that to get notified > problem alerting profile should be Created and hence > Management zone rules should be configured and customized to include all related entities.
   
   
2. The point here is to ensure the K8s configured management zone contains all K8s related entities and then you can easily create metrics and receive alerts upon the specified conditions.

For the K8s related anomaly detections, feel free to customize it as per the K8s admin inputs

BR,

Peter

[GosiaMurawska]

Hi @gurugogi! Did you manage to solve this one? 🤔

[gurugogi · Accepted solution]

Hi Gosia,

Yes I solved it through metric expression

Expression:
builtin:kubernetes.container.restarts
:filter(and(eq("k8s.cluster.name","cluster-name"),eq("k8s.namespace.name","ns")))
:splitBy("k8s.cluster.name","k8s.pod.name","k8s.container.name","k8s.namespace.name")
:sort(value(auto,descending))

Used above expression in Metric events(custom alert)
Descrption:
container Name:{dims:k8s.pod.name}-{dims:k8s.container.name}
Cluster Name:{dims:k8s.cluster.name}
Namespace:{dims:k8s.namespace.name}