topic Re: Question regarding a Metric Event in Container platforms

Question regarding a Metric Event

YuvalKonstanti — Mon, 06 Mar 2023 08:31:12 GMT

Hey Dynatrace Community!

I want to consult with you, regarding a "Metric event" I created.

The purpose: to create an alert, that will be triggered when in our Kubernetes cluster, there are Pods in status pending.

The logic is below:

If you take a look on the image you will see that it seems that the conditions were true for a few occasions, and still no alert was created.

Could you please assist me in understanding if I missed something here?
Thank you!

Re: Question regarding a Metric Event

DanielS — Fri, 03 Mar 2023 13:34:24 GMT

Hi @YuvalKonstanti you can also setup event filters for your pods events.

ie:

Re: Question regarding a Metric Event

florian_g — Tue, 07 Mar 2023 14:00:49 GMT

Hi @YuvalKonstanti are you aware of the K8s alerts already included in Dynatrace? Please read more here: https://community.dynatrace.com/t5/Feedback-channel/Introducing-easy-alerting-on-common-Kubernetes-platform-issues/td-p/197824
Starting with version 1.254, there's also an alert for pending pods.

Re: Question regarding a Metric Event

YuvalKonstanti — Tue, 07 Mar 2023 23:22:21 GMT

Hey,

I believe that you are correct! I do see Dynatrace default problems on this exact issue.

It is still unknown to me, why my custom alert didn't triggered a problem (even though it detect an issue). But for this purpose, I believe Dynatrace does it by default.

Thank you!

Re: Question regarding a Metric Event

florian_g — Thu, 09 Mar 2023 16:52:05 GMT

Regarding your original question - skimming over your screenshots I have some ideas:

did you try to use 0 as threshold? it says "above", so I would assume it's a > and not a >=. Meaning a threshold of 1, would only trigger if you count more than 1.
Which metric event type did you use? I think not all of them create a problem - some only create an event.
Did you allow merging? If yes, maybe your problem got merged with the default problem that Dynatrace created (pending pod alert) and did not create a separate problem? You should see this in the problem card.

Re: Question regarding a Metric Event

YuvalKonstanti — Tue, 28 Mar 2023 09:24:42 GMT

Hey,

1. So firstly, threshold was 5 and I assumed that it is related. I'm guessing that you proposal is correct.

2. That's really interesting! I used "builtin:cloud.kubernetes.workload.pods". How can I know which one is creating an alert and which one isn't?

3. Merging is the option that allow Dynatrace to create "Multiple service problems", isn't it?

Thank you for the answer!