cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Troubleshooting Steps on Memory when the JVM can't even handle Memory Consumption Trending (Instead of Deep Memory Leak Dump)

waikeat_chan
Mentor

Hi all,

I believe the title of my question pretty much says it all.

AFAIK, the memory analysis of AppMon can be tricky and I never done it before. So I decide not to do a memory dump without walking through the documentation and video. Even if so, wouldn't plan to do a Deep Memory Leak first without getting comfortable with Memory Consumption Trending

Anyway, as the memory keep hitting to the max more and more frequent now, I decide to have a schedule/task of perform 'Memory Consumption Trending', couldn't work. Then I try to just do it ONE TIME only, still doesn't work. Whenever I try to initiate a memory dump, the progress stop at around 1% and then the JVM shutdown/crash.

Among all the observations, most interesting thing is the incident even happens several times from this morning til now (which is night time), and today is actually Saturday, which means no way the traffic is heavier than usual. So I wonder what can cause the churn rate to spike up......

Any idea on further troubleshooting?

After the restart of the JVM, I can see that in process overview there only the charts of "passing transaction", "GC-caused suspension time" and "CPU usage" are showing something. Is this a sign of serious memory leak? (can't really understand this as the agent is still manage to be load after the JVM restart, although all the Generation's committed memory is wrong)

4 REPLIES 4

andreas_grabner
Dynatrace Leader
Dynatrace Leader

Hi

I have some resources for you to look at:

* youTube Tutorial on Memory Diagnostics for Java & .NET: https://www.youtube.com/watch?v=SWy83EW_xhk&index=...

* Blog on the same topic: https://www.dynatrace.com/blog/hands-tutorial-5-st...

Andi

waikeat_chan
Mentor

Hi Andi,

Thanks for the links. Now that with everything starts from almost zero again, I would keep on taking trending memory dump and hopefully, able to identify memory leak with or without configure a custom memory sensor.

By the way, another interesting thing I noticed (more of a product feature thing), I notice that whenever I create a task for taking memory dump there is also this entry specifying the process ID of the agent. If the agent is unloaded and then loaded again, the process ID would change and I have to reconfigure/ create a new task for it, since the process ID is already different.

So this explain why back then I can only generate dump manually every hour but would failed everytime I click the task to 'run now'. I was wondering is there any way the configured task can automatically reflect/detect the change in agent's PID?

Regards,

Wai Keat

When creating the task, after you select the agent, it populates the process ID for you.

Did you try to erase the process ID and leave it blank and using an Agent Instance Name (to avoid multiple agent matching)? Remember that if there is a more then one match only the first match will be used.

Gabriel,

Yes, I erase the agent ID and now it is working as expected.

Though I don't know why by default it would populate the PID in it, but anyway, thanks!

Wai Keat