cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Failing theat dump

henk_stobbe
Mentor

Hi,

Does anybody has a comment on below log, it looks like heap dump is busy for to long (with truncate string) and in the meantime the agent failes?

KR Henk


2016-11-09 12:15:12 INFO [AgentCenter] got ResourceExhausted Notification from agent: cmslap1247p_prd_tomcat@cmslap1247p.null:654: Java heap space, flags = 3

2016-11-09 12:15:21 INFO [AgentCenter] got ResourceExhausted Notification from agent: cmslap1247p_prd_tomcat@cmslap1247p.null:654: Java heap space, flags = 3

2016-11-09 12:15:31 INFO [AgentCenter] got ResourceExhausted Notification from agent: cmslap1246p_prd_tomcat@cmslap1246p.null:10108: Java heap space, flags = 3

2016-11-09 12:15:31 INFO [AgentCenter] triggering memory dump upon OOM for cmslap1246p_prd_tomcat@cmslap1246p.null:10108.

2016-11-09 12:15:38 INFO [AgentCenter] got ResourceExhausted Notification from agent: cmslap1246p_prd_tomcat@cmslap1246p.null:10108: Java heap space, flags = 3


2016-11-09 12:15:44 WARNING [HeapDumpProtocol30] truncated StringValue from length 65532 to 55012 characters


2016-11-09 12:15:44 INFO [AgentCenter] got ResourceExhausted Notification from agent: cmslap1246p_prd_tomcat@cmslap1246p.null:10108: Java heap space, flags = 3


2016-11-09 12:15:45 WARNING [HeapDumpProtocol30] truncated StringValue from length 65535 to 55012 characters

2016-11-09 12:15:46 WARNING [HeapDumpProtocol30] truncated StringValue from length 65532 to 55012 characters

2016-11-09 12:15:46 WARNING [HeapDumpProtocol30] truncated StringValue from length 65532 to 55012 characters

2016-11-09 12:15:46 WARNING [HeapDumpProtocol30] truncated StringValue from length 56784 to 55012 characters

2016-11-09 12:15:46 WARNING [HeapDumpProtocol30] truncated StringValue from length 65530 to 55012 characters

2016-11-09 12:15:47 WARNING [HeapDumpProtocol30] truncated StringValue from length 65529 to 55012 characters

2016-11-09 12:15:47 WARNING [HeapDumpProtocol30] truncated StringValue from length 58506 to 55012 characters

2016-11-09 12:15:48 WARNING [HeapDumpProtocol30] truncated StringValue from length 65535 to 55012 characters

2016-11-09 12:15:50 WARNING [HeapDumpProtocol30] truncated StringValue from length 65535 to 55012 characters

2016-11-09 12:15:50 WARNING [HeapDumpProtocol30] truncated StringValue from length 65535 to 55012 characters


2016-11-09 12:15:51 INFO [AgentCenter] got ResourceExhausted Notification from agent: cmslap1246p_prd_tomcat@cmslap1246p.null:10108: Java heap space, flags = 3


2016-11-09 12:16:04 WARNING [Instrumentor] Connection to 'cmslap1247p_prd_tomcat@cmslap1247p.null:654' closed (by EOFException: null)

2016-11-09 12:16:04 WARNING [EventCollector] agent connection 'cmslap1247p_prd_tomcat@cmslap1247p.null:654' closed (prev event: 71, current event: -1).

2016-11-09 12:16:04 WARNING [EventCollector] Connection to agent 'cmslap1247p_prd_tomcat@cmslap1247p.null:654' closed without receiving EOF event. Some data might be missing

2016-11-09 12:16:04 WARNING [MemoryDumpReceiver] Memory Dump failed for agent cmslap1247p_prd_tomcat@cmslap1247p.null:654: com.dynatrace.diagnostics.collector.agentcenter.MemoryDumpReceiver run:161

java.io.EOFException

at java.io.DataInputStream.readInt(DataInputStream.java:392)

at com.dynatrace.diagnostics.core.serialization.buffer.wrapper.DataInputStreamWrapper.readInt(SourceFile:63)

at com.dynatrace.diagnostics.collector.protocol.agent.HeapDumpProtocol30.f(SourceFile:571)

at com.dynatrace.diagnostics.collector.protocol.agent.HeapDumpProtocol30.readHeapDumpEvent(SourceFile:205)

at com.dynatrace.diagnostics.collector.agentcenter.MemoryDumpReceiver.run(SourceFile:106)


2016-11-09 12:16:04 INFO [AgentCenter] got ResourceExhausted Notification from agent: cmslap1246p_prd_tomcat@cmslap1246p.null:10108: Java heap space, flags = 3


2016-11-09 12:16:14 WARNING [HeapDumpProtocol30] truncated StringValue from length 58506 to 55012 characters

2016-11-09 12:16:14 WARNING [HeapDumpProtocol30] truncated StringValue from length 65529 to 55012 characters

2016-11-09 12:16:14 WARNING [HeapDumpProtocol30] truncated StringValue from length 56784 to 55012 characters

2016-11-09 12:16:14 WARNING [HeapDumpProtocol30] truncated StringValue from length 65530 to 55012 characters

2016-11-09 12:16:14 WARNING [HeapDumpProtocol30] truncated StringValue from length 65532 to 55012 characters

2016-11-09 12:16:14 WARNING [HeapDumpProtocol30] truncated StringValue from length 65532 to 55012 characters

2016-11-09 12:16:14 WARNING [HeapDumpProtocol30] truncated StringValue from length 65532 to 55012 characters

2016-11-09 12:16:17 WARNING [HeapDumpProtocol30] truncated StringValue from length 65535 to 55012 characters

2016-11-09 12:16:18 WARNING [HeapDumpProtocol30] truncated StringValue from length 65535 to 55012 characters

2016-11-09 12:16:23 WARNING [HeapDumpProtocol30] truncated StringValue from length 65535 to 55012 characters

2016-11-09 12:16:24 WARNING [HeapDumpProtocol30] truncated StringValue from length 65535 to 55012 characters

2016-11-09 12:16:33 WARNING [HeapDumpProtocol30] truncated StringValue from length 65535 to 55012 characters

2016-11-09 12:16:34 WARNING [EventCollector] agent connection 'cmslap1246p_prd_tomcat@cmslap1246p.null:10108' closed (prev event: 71, current event: -1).

2016-11-09 12:16:34 WARNING [EventCollector] Connection to agent 'cmslap1246p_prd_tomcat@cmslap1246p.null:10108' closed without receiving EOF event. Some data might be missing

2016-11-09 12:16:34 WARNING [Instrumentor] Connection to 'cmslap1246p_prd_tomcat@cmslap1246p.null:10108' closed (by EOFException: null)

2016-11-09 12:16:34 WARNING [MemoryDumpReceiver] Memory Dump failed for agent cmslap1246p_prd_tomcat@cmslap1246p.null:10108: com.dynatrace.diagnostics.collector.agentcenter.MemoryDumpReceiver run:161

java.io.EOFException

at java.io.DataInputStream.readFully(DataInputStream.java:197)

at java.io.DataInputStream.readFully(DataInputStream.java:169)

at com.dynatrace.diagnostics.core.serialization.buffer.wrapper.DataInputStreamWrapper.read(SourceFile:35)

at com.dynatrace.diagnostics.collector.protocol.agent.HeapDumpProtocol30.e(SourceFile:531)

at com.dynatrace.diagnostics.collector.protocol.agent.HeapDumpProtocol30.readHeapDumpEvent(SourceFile:201)

at com.dynatrace.diagnostics.collector.agentcenter.MemoryDumpReceiver.run(SourceFile:106)

4 REPLIES 4

andreas_grabner
Dynatrace Guru
Dynatrace Guru

Never seen that before. maybe you want to open a ticket?

henk_stobbe
Mentor

Hi Andi,

Thx for your reply, fyi

SUPDT-27167

Kr Henk

francois_vangra
Inactive

Hi Andreas,

Was there a solution to this issue? Because I'm seeing the same thing now, except with the java agent for JBoss on Solaris.

Regards,

Francois

henk_stobbe
Mentor

Francois,

Sorry,

Ticket was closed: No data provided for over a week. Assuming problem resolved or no longer important

You can open a new ticket, support will need below info:

which operating system is the agent running on?

  • JVM vendor and version
  • appserver version
  • the JVM startup parameter
  • the collector log & server log
  • And gave me this link:

    https://community.dynatrace.com/community/display/DTKB/Known+incompatibilities+with+JVM+versions%2C+...

    KR Henk