Summary: This post shows how to set up alerting rules in Dynatrace to ignore exceptions based on specific message patterns. It includes wildcard usage examples and best practices for reducing noise in alerts.
I have an exception that I am trying to tell Dynatrace to ignore, which is turning out to be tricky.
A couple of questions that I am hoping I can find answers to.
Question 1:
This looks like 2 exceptions, looking at the purepath. You have the top which is…
I am 99% sure the exception to be placed in the “exception class” rule below should be what’s called out as the exception and not the exception listed at the start of the message, correct?
Question #2
We do not want to ignore all exception messages that can come from that exception class. We only want a specific type, which is shown below. Does the message value to filter on start at the “com.altassian.bitbucket.content.NoSuchPathExcetion:” or does it begin at “The path…”?
Message:
com.atlassian.bitbucket.content.NoSuchPathException: The path “Jenkinsfile” does not exist at revision “ecfb6ba40XXXXXXXXXXXXXX847cb3b”
Question #3
Question 2 then takes me into question 3 This specific exception message has a different revision value each and every time. I have tried setting the “type a string that, if found…” by just adding part of the message, but that does not appear to work. For example if I use the value "The path “Jenkinsfile” does not exist at revision" in that field below. It’s a bit confusing because its stated in a way that makes it sound like you can just put part of the message to act as a type of wildcard. Which is also another question… Can I wildcard the revision value part of the message?
To date, I do not think it is possible to wildcard unfortunately. However, yes you can set to ignore exceptions on a global level from one spot. See below…
if you ignore the exceptions in the deep monitoring settings then they are not captured and you won’t see them. This setting is global.
If you change the configuration on the service settings screen, this affects the failure detection - that means exceptions will be captured (you will see them), but they won’t be used as failure reasons. Or you can also setup different rules such as mark request as successful when an exception is found..
Luckily, you don’t have iterate over a list of your services and you can setup rules using API to define those. This is achieved through Failure detection configuration API.
It’s available from Dynatrace version 1.216. This approach is very flexible and allows you to configure rules on subsets of your services.
To summarize:
Set exception to ignore only when you don’t want to capture them - this also has effect on the agent overhead - lower number of exceptions results into lower overhead.
Set failure detection rules or service failure detection when you want to see the exception, but not mark those requests as failed.
From my experience, the failure detection mechanism is often overlooked by customers and then they are struggling with alerting noise since Dynatrace considers unknown exceptions to be failures by default. (Exceptions messages such as “Bad password” are good examples).
My need is to put that rules to ignore more than 500 exceptions (business exceptions: the team gived me a list of message--> dimension Message) mainly to not generate alerts.
So do you have an explicite example to show how can I create a list of parameter based on exception and dimension message, after that create a rule to put.
Also, after creating the rule, it will appear in the global setting under this menu ?
@Malaik No, those rules are not visible in the UI at the moment. Unfortunately.
Your screenshot shows exceptions that will be excluded from capturing. This is different from the exceptions ignored for failure detection described below.
You need a parameterSet - those parameters are applied
{
"id": "My_Business_Exceptions",
"name": "My Business exceptions handling",
"description": "Rules for ignoring exceptions in my awesome business application",
"successForcingExceptionPatterns": [
],
"ignoredExceptionPatterns": [
{
"classPattern": "org.springframework.security.access.AccessDeniedException",
"messagePattern": ""
}
]
}
Then you need detection rule describing where to apply the ruleset (fdpId must be equal to the parameter set id created above):
Agreed. At least the rules should be visible when they are effective on a service. Just as it is with the global naming rules (also editable using API only).