Summary
HTTP status code alerting using different options
Problem
In some cases, alerts are not wanted for failure rates that have a specific response status code and/ or exception, but misconfiguration can also mean that alerts unexpectedly occur. Below are some scenarios for defining the Failure detection rule.
Resolution
Failure detection rule using Global Settings
This documentation explains how to set up failure detection.
- Now let’s understand if we only want to get Failure rate alert for specific response code(server side or Client side) define the response code as below screenshot.
If you see above example, I have defined status code for which I only want failure rate should detect. For the missing or not defined status will be skipped or will not include in failure.
Note: In both HTTP Response Code fields, you can also add 4xx or 5xx codes only if you see that your request calls have those errors in Server side or Client side.
- There are some cases where you are still getting Failure rate because the Status code is having an exception, which should also be either IGNORED or should be treated as SUCCESS. For that, you would need to define the list of exceptions that should not be a failure. You can add the exceptions in either of the configurations.
- After you have defined the Parameters, next would be to set the Failure Detection rule which is important when applying it in global settings. If this is not applied, then Global settings will not be applied.
- For this, you would need to go to Settings---> Server-Side Service Monitoring----> Failure Detection Rule. In that you need to first add the Failure Detection Parameter which you have created, and then you need to add the Conditions(second screenshot)
- Based on the above conditions, the Failure detection rule will be applied. This is helpful when you only want to apply to a specific set of services either using TAG, MZ or PG name, etc.
- Once it's applied based on the condition, you will see that this rule is also applied into those specific services.
Few example from the traces where it shows Failure rate before and after applying.
Before Failure rate detection Rule:
After Failure rate detection rule:
Failure rate detection on Service level
This documentation explains how to setup failure detection on Service level.
- When setting Failure detection rule on Service level, the Global Rule will not be applied here for HTTP status code as well as General Parameter(Exceptions).
- Now the general use case would be to apply the Failure detection rule on service similar to Global settings, but you might want that HTTP status code should be applied as per Global level or Exception should apply from Global rule (EITHER/OR).
- For this, you need to make sure that “OVERRIDE GLOBAL FAILURE DETECTION SETTINGS” is either Turned ON or OFF as per your requirements. Based on this conditions, the rules will get applied.
- Rest the HTTP response status code and Exception can be defined similar to Global settings.
What's next
If this article did not help, please open a support ticket, mention that this article was used and provide the following in the ticket:
- Tenant/Environment Link with affected service details
