cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Java Exception Sensor Configuration

miteglia
Participant

Hi Everyone,

I'm trying to have Dynatrace detect application errors in Apache NiFi. And to do this I believe I need to configure capture settings for Java Exceptions.

The thing is I'm not yet sure how Apache NiFi throws errors, or what specific Java Exceptions I need to use...

If anyone has configured Java Exceptions for error derection for a Java-based application, please share how you did it and the exact exception, method, class, etc. you used.

One last thing: what generic methods and/or class used to capture errors or exceptions in Java can I use? The goal is for Dynatrace to capture any kind of error in the application.

Thank you in advance for the feedback.

17 REPLIES 17

dave_mauney
Dynatrace Champion
Dynatrace Champion

Hi,

We capture most exceptions by default with no action required on your part. There is an "Exceptions Sensor" that must be placed on your NiFi agent group for it to work. You can also select "Sensor Configuration" in your agent group and modify its properties to choose exceptions to exclude or to include only selected exceptions (this is rarely used, I think). But we already instrument the catch statements in java code to pick up most exceptions for you. I say most because some are ignored by the Exceptions Sensor to avoid too much noise.

HTH,

dave

Thank you very much for the answer Dave.

However I find your first 2 statements conflicting. You say exceptions are captured by default, but then also say that an exceptions sensor must be configured for it to work... Could you please elaborate on placing the sensor in the agent group? By this do you mean the Exceptions Sensor Pack in the Sensor Configuration, or the Unhandled Exceptions Rules in Error Detection?

Also if DT already picks up most of the exceptions, where can I see them? And how can I configure them to show on a dashboard, or even more send an email to a specified group?

Again, thanks for your feedback, much appreciated! 🙂

What Dave means is that out-of-the-box, we ship a configuration which will capture most exceptions. Whenever you create a new agent tier, you need to ensure that this sensor pack (yes, in the sensor configuration) is placed on that particular tier.

However, if you want to capture every exception apart from your com.dynatrace.whatever we also provide that flexibility.

If you look in the left hand menu (sometimes referred to as the cockpit), you'll see an Exceptions view (we call these views dashlets). If you open it, you'll see a fullscreen view of all the exceptions in the system.

If you want to create your own dashboard you can customise the dashlets on the dashboard ... For example, you could add a chart and an exceptions dashlet side-by-side or two charts, an exceptions dashlet and a transaction flow dashlet.

Once you have a dashboard set up the way you want, save it to the server and this can then be used as the backbone for an automated report. Just right click the dashboard name and choose Report

andreas_grabner
Dynatrace Guru
Dynatrace Guru

FYI - here is the content of a side-conversation Michele and I had via email about the same topic. I want to make sure all this information is available in one spot:

It
seems that your initial question was answered on the forum in the meantime. I
think you should watch some of my Advanced YouTube Tutorials that explain how
PurePath technology and our Sensors work – that might explain some of the
things you see

When
you place the Exception Sensor it means we are instrumenting all Exception
classes that are defined by that Sensor Pack. That list of classes is WITHIN
that Sensor Pack that is globally installed on the Dynatrace Server. If you
want to see the list of classes go to Settings -> Dynatrace Server -> Sensor
Packs and edit the Java Exception Sensor.

What
this means is that we are instrumenting a lot of Exception Classes but we then
only capture information for the PurePAth based on the configuration in your
Exception Sensor Properties. You can for instance say: Hey Dynatrace – I know
you instrument java.xxx.Exception and all its derived classes – but – I really
only want to see com.mycode.myexceptions and not the rest”. These capture
settings are configured through the Sensor Properties.

The
same is true for other sensors, e.g: the JDBC Sensor instruments core JDBC
Methods but through the properties you can specify whether you want to capture
just SQL Statements or also bind values.

Thanks for the vids, Andi. They're truly helpful. +1.

miteglia
Participant

dynatrace-screenshots-apr22.docx

Hi Everyone,

Thank you for all the feedback, things are clearer now. Now I'm in the process of defining the Java Exceptions, and new concerns came up that I would ask for your direction please:

  • 1.Unable to edit the placement of and
    exclusion/inclusion of method sensors in the System Profiles -> Manage Server...
    -> Sensor Packs, the check boxes are greyed out and disabled. I restarted dynatrace
    and the application, but still the same
  • 2.I think I’m seeing Exceptions in the Deployed
    Sensors, but why aren’t they appearing in Exceptions Dashlet? Are the
    Exceptions in the Deployed Sensors and Exceptions dashlet different?
  • 3.I’ve removed all defined Exceptions in Sensor
    Packs and Sensor Configuration, but I still get results in Deployed Sensors. I think
    this is because I couldn’t remove the placement of sensors in the Sensors
    Packs?
  • 4.May I please know the difference between 2 areas
    where Sensors are configured? In System Profiles -> Manage Server.. ->
    Sensor Packs -> Exceptions, versus Monitoring -> Edit System Profile…
    -> Sensor Configuration -> Exceptions.
  • 5.Could you please provide best practices in defining
    Java Exception Sensors? Basically we need to see any and all kinds of error in
    NiFi, do you think below configuration should do the job?

Exclude all but include these

java.lang.Exception. = starts

java.lang.Error. = starts

java.lang.Throwable. = starts

Error. = contains

java. = contains

The above Exceptions are the ones I configured in the Sensor Configuration. But still, no Exceptions are getting picked up....

Please find detailed screenshots in the attached document. I truly appreciate your guidance and direction. Thank you.

Sincerely,

Micky

Hello Micky,

To achieve the following you can go into 'DEBUG' client mode with the 'Ctrl + Shift +F9).

Unable to edit the placement of and exclusion/inclusion of method sensors in the System Profiles -> Manage Server... -> Sensor Packs, the check boxes are greyed out and disabled. I restarted dynatrace and the application, but still the same.

Java. class is globally excluded from the 'Exception' sensor properties by default.

Sensor Packs are configured on the appmon/server level and the 'Sensors' can be configured for a specific system profile.

May I please know the difference between 2 areas where Sensors are configured? In System Profiles -> Manage Server.. -> Sensor Packs -> Exceptions, versus Monitoring -> Edit System Profile… -> Sensor Configuration -> Exceptions.

Regards,

Babar

Hello Babar,

Thanks very much for the prompt response. I'm now in debug mode.

Basically I want to catch all Java Exceptions. Please correct me if wrong. My understanding is that "Place" should be checked, then "Include=Active" means the Exceptions will be captured. So I should make the "Exclude" to Inactive, because if Exclude is Active, the exception will not be captured.

But how can I tag a sensor as "include"? Because some are tagged as "exclude" by default...

Also with your second answer, so you mean I need to change the Sensor Packs first, then the Sensor in the system profile? So changes in the Sensor Packs will affect the Sensors configured in the System Profile, and vice versa?

Thank you!

Hello Micky,

Your understanding about 'Place', 'Exclude', and 'Inactive', etc... You can even change the 'Exclude' to 'Include' but it will take effect after restart the application or hot sensor placement.

As I shared before that java.is globally excluded for exceptions sensor.

Regards,

Babar

Hello Babar,

I've removed that java. lang exception so that should not matter anymore.

miteglia
Participant

Just an additional inquiry please:

What additional configuration must be done in order to see a Purepath? I thought purepaths will appear by default once the application is running and detected by DT...

In the monitoring dashboard, it says: "No transactions for the selected timeframe. Ensure that your application is under load, generate traffic or adapt the selected timeframe."
Could you please elaborate on how to overcome this and have the purepath/s show themselves?
My application is running on my local machine where Dynatrace is also installed. So DT is only connected to one application tier which is NiFi. In NiFi which is a dataflow application, only one simple dataflow is configured, consisting of 5 processors.

Could it be something "too simple" (no high data loads, no UEM, no webservers, etc.) for Dynatrace to make a purepath for?

Thank you very much.

Hello Micky,

In the first stage you can check two following things:

  • Open the 'Agents Overview' dashboard and check the 'Event Count', 'Class Load Count', 'Total PurePath Count' columns, whether you have some number or not.
  • Open the 'Sensors Configuration' of agent group and check the relevant sensors are 'active and start Pureapths' from the especially 'Web Server'

Regards,

Babar

Hello Babar,

Here are the details:

Event Count = 19

Class Load Count= 2555

Total PurePath Count = 0

For the Java Exception sensor, it only says Inactive and Active. There is no Active and start PurePaths. Still, I've set it to Active.

Still, I cannot see PurePaths in the Monitoring dashboard...

Hello Micky,

Exception sensor is nothing to do with the PurePaths. Check the following sensors for the 'active and start PurePaths', and also active the 'Java Web Requests':

  • Java Web Services
  • Servlets

If nothing works then may be we will have to create a custom sensor as an entry point.

Regards,

Babar

Hello Babar,

My application is Java-based and does not deal with Java Web Services nor Servlets. Ok so exception sensors have nothing to do with purepaths, but could you please expound on why am I not seeing PurePaths still?

Thank you.

miteglia
Participant

Here is Andi's reply from our email conversation:

Hi Michele

Before I answer – please make sure that we also post this on the community as otherwise we have information floating around in different places. This is a great learning experience not just for us but for all Dynatrace users out there.

First of all I have to say that I am NOT a fan of modifying our Out-of-the-Box Sensor Pack Configuration. There is a reason why we typically exclude certain methods or in this case certain Exceptions GLOBALLY in the Exception Sensor Definition. Reason is that there are a lot of core Java Exceptions that are thrown by an application. Most of them are internally to the JVM or other core frameworks. If we start capturing ALL these exceptions it could potentially have an impact on the overhead we cause by capturing all that information. In most cases these exceptions are also not that valuable when analyzing errors in your own custom code as you typically use either your own exception classes or exceptions that derive from these core classes we excluded.

Having all this said – let me try to explain how sensors work. I think I also cover this in my YouTube Tutorials on Advanced Sensors – but here is I quick Summary

#1: Sensor Packs and Sensor Rules

Sensor Packs – whether the global ones you see in the Server Settings -> Sensor Packs or those that you can define in your System Profile under Sensors tell the Dynatrace Agent which Methods to instrument. This means we actually put some custom code in these classes so that we are able to capture method execution. The Java Sensor Pack defines a list of Exception Classes but also excludes some to be sure we only instrument those that really make sense to be instrumented as we assume they make sense for our users to capture. Sensors also have the capbilty to say “I want to instrument this class or interface and all derived classes” -> this actually makes it possible to define very generic Sensor Rules to say e.g: java.lang. Exception as this would not only instrument java.lang.Exception but also all derived classes. If we have such a generic rule but want to exclude certain specific classes along the class hierarchy we can additional define exclusion rules, e.g: we can say that we want to include java.lang.Excption and all its derived classes BUT WE WANT TO EXCLUDE java.lang.Exception itself and maybe some others.

When an application starts and our Agent gets loaded into the Class we intercept all loaded classes. Every class that gets loaded gets analyzed and we match every method against ALL Placed Sensor Packs. If there is a rule that wants to instrument that method then we instrument it. That’s also true for the Exceptions and the rules that come in through the global Java Exception Sensor.

#2: Sensor Properties

For certain Sensors we do not just instrument and capture. We additionally allow you on a per Agent Group level property configuration to decide which Exceptions to really capture when we see an Exception object being created as part of a PurePath. You can also tell the Exception Sensor on whether to capture Full Stack Traces, how many of those and how deep. These are all properties that influence the code we instrument into the Exception Constructor itself of all these Exceptions that are defined in the Sensor Pack.

Why you might still not see Exceptions even though you see them in the Deployed Sensors and you have not execluded them in the Properties?

Remember – Dynatrace always captures data in the context of a PurePath. That means – we will only capture Method Executions, Return Values, Database Queries, Web Serivce Calls … AND Exceptions – in case these things are executed in the context of a currently captured PurePath. A PurePath is the core concept of Dynatrace. It is our capability to do end-to-end transaction tracing. A PurePath always starts at an Entry Point, e.g. Servlet or Web Service entry Point. Once we trace a PurePath we start capturing additional data from all placed sensors.

There can be two reasons why you don’t see the exception that you expect

#1: The Exceptions are thrown outside the context of a PurePath, e.g: in a background thread or some other transacton that is currently not captured by a PurePath. If that is the case you can define custom entry point sensors that will start a new PurePath for your specific Background Threads, Batch Jobs or custom frameworks that dynatrace doesn’t understand out-of-the-box.

#2: PurePaths are normally automatically traced across runtime and thread boundaries. That is TRUE if we support e.g: your used Thread Pool Implementation. If we do not support your custom Thread Pool it could mean that we do not automatically follow the PurePath in all your background threads and therefore also not able to see your exceptions. This is a tougher case because that might require some manual instrumentation or involvement of our engineering team to build support for your thread pool

I realize that this is a lot of information. I hope it didn’t raise more questions than it answered.

Andi

matthias_dillie
Organizer

I have a additional question.

Is it possible to use more detailed definitions of which exceptions are captured, e.g. with regex. I would like to exclude all exceptions beginning with 'java.' but also include the specific exception '

java.lang.IllegalStateException'.

Any ideas or do I have to add a request for enhancements?