cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Custom Alert Not Firing When Using Code-based Metric Query

swin12
Participant

Hi,

 

I am trying to configure custom alerts for a service's processing time. These alerts are based on a calculated service metric I created that allows me to filter request data by certain dimension; request attributes.

 

I have configured this alarm end-to-end and gotten an alert to successfully fire under the condition I configure the alert using the 'build' option (filtering by only a single dimension value):

 

swin12_1-1656607461563.png

swin12_2-1656607540101.png

 

When I try to create a very similar alert using the 'code' option (because I need to filter on multiple values for the dimension, which doesn't seem possible using the 'build' option), I get a preview that indicates alerts should fire, but they never actually fire:

 

swin12_0-1656607450089.png

swin12_3-1656607858215.png

 

The settings for the two alarms are identical, in that:

 

  • They have identical static threshold settings
  • They have identical severity settings (Slowdown)

 

Where they differ is:

  • I used the 'build' option for one and the 'code' option for the other.
  • For the alert that uses the 'build' option, I am able to set entity scopes for the alert (such as management zone).
    swin12_4-1656608492855.png
  • For the alert that uses the 'code' option, I am unable to set these entity scopes (i.e. I can't select management zone from the dropdown).

    swin12_5-1656608530513.png

 

My leading theory is that the issue lies with the fact that the code-based alert does not have a management zone entity set (it doesn't seem to let me set one). As a result, the alert doesn't match the management zone for my alerting profile, and therefore I don't get an alert.

 

How would I associate a code-based alert with a specific management zone, such that it is picked up by the alerting profile when the alarm fires? Is there something I can add to the metric query?

1 REPLY 1

swin12
Participant

The problem was that the custom alert wasn't related to the management zone condition that I had configured in my alerting profile.

 

I ended up removing the management zone condition from the alerting profile, instead relying on a different condition on the alerting profile; matching on the name of the custom alert. Once I did this, the alert starting sending notifications as expected.

 

I still don't fully understand which conditions cause a custom alert to be associated with a management zone (such that it is 'respected' by an alerting profile that has a management zone condition set). My best guess is that you must have a management zone entity filter applied in the custom alert configuration. I still don't understand how this is possible when the custom alert uses a code-based metric expression (as you seem to lose the ability to set a management zone entity filter this way).