cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

This product reached the end of support date on March 31, 2021.

problems with "Potentially stuck Java VM detected" rule incident

pbrown
Guide

hello!

I'm currently using 6.3.7.1007 DT version.

I recently configured an incident rule to address this stuck JMV problem:

but even I've got some incident rules triggered, no thread dump has been made for today 08/31:

Am I missing something or maybe looking for threads dumps where I will not find them?

thanks!!

Patricio

8 REPLIES 8

david_n
Inactive

Hello Patricio,

It could be that your dumps are being overwritten in the session store. When the session store is starting to fill up Dynatrace will remove the content that is not locked in order to create space for new session data (Purepaths, Memory/Thread Dumps and CPU Samples). I would suggest locking your sessions that are created via this incident (boolean checkbox in the settings). I also suggest checking you storage settings to make sure that your session store is not in a constantly full state. You can do that by going to Settings -> Dynatrace Server -> Storage.

Thanks,

David Nicholls

Hi David,

thanks for the tip. I'll do as you said.

regarding to storage, I checked it everyday and there is always at least 129 GB free.

thanks

Patricio

Adding to this I would recommend checking the server logs just to verify that it is actually attempting to create a thread dump when the incident triggers.

pbrown
Guide

Hi James,

I found this in the log !com.dynatrace.diagnostics.plugin.ThreadDumpAction.action.0.0.log"

what does it mean? thanks!

2016-08-30 12:39:02 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Resource dump creation triggered by incident rule Potentially stuck Java VM detected
2016-08-30 15:48:12 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Previous message was repeated 7 times.
2016-08-30 15:48:12 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Resource dump creation triggered by incident rule Potentially stuck Java VM detected
2016-08-30 16:10:22 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Previous message was repeated 16 times.
2016-08-30 16:10:22 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Resource dump creation triggered by incident rule Potentially stuck Java VM detected
2016-08-30 16:43:12 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Previous message was repeated 7 times.
2016-08-30 16:43:12 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Resource dump creation triggered by incident rule Potentially stuck Java VM detected
2016-08-30 17:52:12 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Resource dump creation triggered by incident rule Potentially stuck Java VM detected
2016-08-30 18:31:52 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Previous message was repeated 15 times.
2016-08-30 18:31:52 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Resource dump creation triggered by incident rule Potentially stuck Java VM detected
2016-08-31 09:58:22 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Previous message was repeated 7 times.
2016-08-31 09:58:22 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Resource dump creation triggered by incident rule Potentially stuck Java VM detected
2016-08-31 10:30:12 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Resource dump creation triggered by incident rule Potentially stuck Java VM detected
2016-08-31 10:46:12 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Previous message was repeated 37 times.
2016-08-31 10:46:12 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Resource dump creation triggered by incident rule Potentially stuck Java VM detected
2016-08-31 12:19:42 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Previous message was repeated 4 times.
2016-08-31 12:19:42 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Resource dump creation triggered by incident rule Potentially stuck Java VM detected
2016-08-31 14:10:32 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Previous message was repeated 6 times.
2016-08-31 14:10:32 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Resource dump creation triggered by incident rule Potentially stuck Java VM detected
2016-08-31 15:42:12 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Previous message was repeated 5 times.
2016-08-31 15:42:12 INFO [AbstractDumpAction@com.dynatrace.diagnostics.plugin.ThreadDumpAction.action] Resource dump creation triggered by incident rule Potentially stuck Java VM detected

Hi,

could you find the reason why it is not generating thread dump?

My opinion is that as the JVM is in a stuck state the dynatrace agents also wouldn't be able to take any runtime analysis.

pbrown
Guide

Hi Melory,

yes indeed, after several test and log checking with the support team, they reach the same conclusion you told me. The JMV gets stuck and prevent the agent from generating the thread dump.

thanks!

PBA

Hello There Appmon-ites,

I have a quick query: What is the incident "Potentially stuck Java VM" signifies?

AFAIK, the details indicate:

Potentially stuck Java VM. It's neither due to a heap/thread dump or GC activity. The reason is unknown and requires manual investigation.

1. Could anyone pl. explain whether the JVM (java.exe or java) is stuck on OS level or does it means that all the threads in app servers are hung?

2. Also what is difference between hung threads vs the Stuck JVM?

3. Will taking Thread Dump or/and heap dump should be helpful to analyze the cause?

4. Can I exploit the Generic Execution Plugin or custom PDK plugin to trigger the thread dumps on JVM?

Pl. help,

Regards, Rajesh.