I'm trying to have Dynatrace detect application errors in Apache NiFi. And to do this I believe I need to configure capture settings for Java Exceptions.
The thing is I'm not yet sure how Apache NiFi throws errors, or what specific Java Exceptions I need to use...
If anyone has configured Java Exceptions for error derection for a Java-based application, please share how you did it and the exact exception, method, class, etc. you used.
One last thing: what generic methods and/or class used to capture errors or exceptions in Java can I use? The goal is for Dynatrace to capture any kind of error in the application.
Thank you in advance for the feedback.
We capture most exceptions by default with no action required on your part. There is an "Exceptions Sensor" that must be placed on your NiFi agent group for it to work. You can also select "Sensor Configuration" in your agent group and modify its properties to choose exceptions to exclude or to include only selected exceptions (this is rarely used, I think). But we already instrument the catch statements in java code to pick up most exceptions for you. I say most because some are ignored by the Exceptions Sensor to avoid too much noise.
Thank you very much for the answer Dave.
However I find your first 2 statements conflicting. You say exceptions are captured by default, but then also say that an exceptions sensor must be configured for it to work... Could you please elaborate on placing the sensor in the agent group? By this do you mean the Exceptions Sensor Pack in the Sensor Configuration, or the Unhandled Exceptions Rules in Error Detection?
Also if DT already picks up most of the exceptions, where can I see them? And how can I configure them to show on a dashboard, or even more send an email to a specified group?
Again, thanks for your feedback, much appreciated! 🙂
What Dave means is that out-of-the-box, we ship a configuration which will capture most exceptions. Whenever you create a new agent tier, you need to ensure that this sensor pack (yes, in the sensor configuration) is placed on that particular tier.
However, if you want to capture every exception apart from your com.dynatrace.whatever we also provide that flexibility.
If you look in the left hand menu (sometimes referred to as the cockpit), you'll see an Exceptions view (we call these views dashlets). If you open it, you'll see a fullscreen view of all the exceptions in the system.
If you want to create your own dashboard you can customise the dashlets on the dashboard ... For example, you could add a chart and an exceptions dashlet side-by-side or two charts, an exceptions dashlet and a transaction flow dashlet.
Once you have a dashboard set up the way you want, save it to the server and this can then be used as the backbone for an automated report. Just right click the dashboard name and choose Report
FYI - here is the content of a side-conversation Michele and I had via email about the same topic. I want to make sure all this information is available in one spot:
seems that your initial question was answered on the forum in the meantime. I
think you should watch some of my Advanced YouTube Tutorials that explain how
PurePath technology and our Sensors work – that might explain some of the
things you see
you place the Exception Sensor it means we are instrumenting all Exception
classes that are defined by that Sensor Pack. That list of classes is WITHIN
that Sensor Pack that is globally installed on the Dynatrace Server. If you
want to see the list of classes go to Settings -> Dynatrace Server -> Sensor
Packs and edit the Java Exception Sensor.
this means is that we are instrumenting a lot of Exception Classes but we then
only capture information for the PurePAth based on the configuration in your
Exception Sensor Properties. You can for instance say: Hey Dynatrace – I know
you instrument java.xxx.Exception and all its derived classes – but – I really
only want to see com.mycode.myexceptions and not the rest”. These capture
settings are configured through the Sensor Properties.
same is true for other sensors, e.g: the JDBC Sensor instruments core JDBC
Methods but through the properties you can specify whether you want to capture
just SQL Statements or also bind values.
Thank you for all the feedback, things are clearer now. Now I'm in the process of defining the Java Exceptions, and new concerns came up that I would ask for your direction please:
Exclude all but include these
java.lang.Exception. = starts
java.lang.Error. = starts
java.lang.Throwable. = starts
Error. = contains
java. = contains
The above Exceptions are the ones I configured in the Sensor Configuration. But still, no Exceptions are getting picked up....
Please find detailed screenshots in the attached document. I truly appreciate your guidance and direction. Thank you.
To achieve the following you can go into 'DEBUG' client mode with the 'Ctrl + Shift +F9).
Unable to edit the placement of and exclusion/inclusion of method sensors in the System Profiles -> Manage Server... -> Sensor Packs, the check boxes are greyed out and disabled. I restarted dynatrace and the application, but still the same.
Java. class is globally excluded from the 'Exception' sensor properties by default.
Sensor Packs are configured on the appmon/server level and the 'Sensors' can be configured for a specific system profile.
May I please know the difference between 2 areas where Sensors are configured? In System Profiles -> Manage Server.. -> Sensor Packs -> Exceptions, versus Monitoring -> Edit System Profile… -> Sensor Configuration -> Exceptions.
Thanks very much for the prompt response. I'm now in debug mode.
Basically I want to catch all Java Exceptions. Please correct me if wrong. My understanding is that "Place" should be checked, then "Include=Active" means the Exceptions will be captured. So I should make the "Exclude" to Inactive, because if Exclude is Active, the exception will not be captured.
But how can I tag a sensor as "include"? Because some are tagged as "exclude" by default...
Also with your second answer, so you mean I need to change the Sensor Packs first, then the Sensor in the system profile? So changes in the Sensor Packs will affect the Sensors configured in the System Profile, and vice versa?
Your understanding about 'Place', 'Exclude', and 'Inactive', etc... You can even change the 'Exclude' to 'Include' but it will take effect after restart the application or hot sensor placement.
As I shared before that java.is globally excluded for exceptions sensor.
Just an additional inquiry please:
What additional configuration must be done in order to see a Purepath? I thought purepaths will appear by default once the application is running and detected by DT...
In the monitoring dashboard, it says: "No transactions for the selected timeframe. Ensure that your application is under load, generate traffic or adapt the selected timeframe."
Could you please elaborate on how to overcome this and have the purepath/s show themselves?
My application is running on my local machine where Dynatrace is also installed. So DT is only connected to one application tier which is NiFi. In NiFi which is a dataflow application, only one simple dataflow is configured, consisting of 5 processors.
Could it be something "too simple" (no high data loads, no UEM, no webservers, etc.) for Dynatrace to make a purepath for?
Thank you very much.
In the first stage you can check two following things:
Here are the details:
Event Count = 19
Class Load Count= 2555
Total PurePath Count = 0
For the Java Exception sensor, it only says Inactive and Active. There is no Active and start PurePaths. Still, I've set it to Active.
Still, I cannot see PurePaths in the Monitoring dashboard...
Exception sensor is nothing to do with the PurePaths. Check the following sensors for the 'active and start PurePaths', and also active the 'Java Web Requests':
If nothing works then may be we will have to create a custom sensor as an entry point.
My application is Java-based and does not deal with Java Web Services nor Servlets. Ok so exception sensors have nothing to do with purepaths, but could you please expound on why am I not seeing PurePaths still?
Here is Andi's reply from our email conversation:
Before I answer – please make sure that we also post this on the community as otherwise we have information floating around in different places. This is a great learning experience not just for us but for all Dynatrace users out there.
First of all I have to say that I am NOT a fan of modifying our Out-of-the-Box Sensor Pack Configuration. There is a reason why we typically exclude certain methods or in this case certain Exceptions GLOBALLY in the Exception Sensor Definition. Reason is that there are a lot of core Java Exceptions that are thrown by an application. Most of them are internally to the JVM or other core frameworks. If we start capturing ALL these exceptions it could potentially have an impact on the overhead we cause by capturing all that information. In most cases these exceptions are also not that valuable when analyzing errors in your own custom code as you typically use either your own exception classes or exceptions that derive from these core classes we excluded.
Having all this said – let me try to explain how sensors work. I think I also cover this in my YouTube Tutorials on Advanced Sensors – but here is I quick Summary
#1: Sensor Packs and Sensor Rules
Sensor Packs – whether the global ones you see in the Server Settings -> Sensor Packs or those that you can define in your System Profile under Sensors tell the Dynatrace Agent which Methods to instrument. This means we actually put some custom code in these classes so that we are able to capture method execution. The Java Sensor Pack defines a list of Exception Classes but also excludes some to be sure we only instrument those that really make sense to be instrumented as we assume they make sense for our users to capture. Sensors also have the capbilty to say “I want to instrument this class or interface and all derived classes” -> this actually makes it possible to define very generic Sensor Rules to say e.g: java.lang. Exception as this would not only instrument java.lang.Exception but also all derived classes. If we have such a generic rule but want to exclude certain specific classes along the class hierarchy we can additional define exclusion rules, e.g: we can say that we want to include java.lang.Excption and all its derived classes BUT WE WANT TO EXCLUDE java.lang.Exception itself and maybe some others.
When an application starts and our Agent gets loaded into the Class we intercept all loaded classes. Every class that gets loaded gets analyzed and we match every method against ALL Placed Sensor Packs. If there is a rule that wants to instrument that method then we instrument it. That’s also true for the Exceptions and the rules that come in through the global Java Exception Sensor.
#2: Sensor Properties
For certain Sensors we do not just instrument and capture. We additionally allow you on a per Agent Group level property configuration to decide which Exceptions to really capture when we see an Exception object being created as part of a PurePath. You can also tell the Exception Sensor on whether to capture Full Stack Traces, how many of those and how deep. These are all properties that influence the code we instrument into the Exception Constructor itself of all these Exceptions that are defined in the Sensor Pack.
Why you might still not see Exceptions even though you see them in the Deployed Sensors and you have not execluded them in the Properties?
Remember – Dynatrace always captures data in the context of a PurePath. That means – we will only capture Method Executions, Return Values, Database Queries, Web Serivce Calls … AND Exceptions – in case these things are executed in the context of a currently captured PurePath. A PurePath is the core concept of Dynatrace. It is our capability to do end-to-end transaction tracing. A PurePath always starts at an Entry Point, e.g. Servlet or Web Service entry Point. Once we trace a PurePath we start capturing additional data from all placed sensors.
There can be two reasons why you don’t see the exception that you expect
#1: The Exceptions are thrown outside the context of a PurePath, e.g: in a background thread or some other transacton that is currently not captured by a PurePath. If that is the case you can define custom entry point sensors that will start a new PurePath for your specific Background Threads, Batch Jobs or custom frameworks that dynatrace doesn’t understand out-of-the-box.
#2: PurePaths are normally automatically traced across runtime and thread boundaries. That is TRUE if we support e.g: your used Thread Pool Implementation. If we do not support your custom Thread Pool it could mean that we do not automatically follow the PurePath in all your background threads and therefore also not able to see your exceptions. This is a tougher case because that might require some manual instrumentation or involvement of our engineering team to build support for your thread pool
I realize that this is a lot of information. I hope it didn’t raise more questions than it answered.
I have a additional question.
Is it possible to use more detailed definitions of which exceptions are captured, e.g. with regex. I would like to exclude all exceptions beginning with 'java.' but also include the specific exception '
Any ideas or do I have to add a request for enhancements?