Tomcat process memory higher than the JVM heap memory

[naveench]

We are running a SpringBoot application in tomcat 9.x with OpenJDK11. This application is hosted on a VM.

We have been observing high memory utilization of the tomcat process in dynatrace. "Memory Usage" graph of the process is showing 5.3gb while the heap memory usage is 2 gb. Can someone help me where the remaining 3.3gb memory is used?

Note: tomcat profile contains only one application running in it. We are using Xms as 1024m and Xmx as 4096.

[richard_guerra]

Observation - in your heap memory chart, I don't see any garbage collection activity. 

Guess - some stuck thread that cascades in the application to build up threads/stack-memory-consumption - with a overall process "hang" at 10:30.

You need a thread dump or heap dump to analyze what is in there that may be the culprit.

If this is hard to do in retrospect, is there a possibility to load-test the app in a lower environment (PTE), and analyze threads and heap objects there? 

Here are two good discussions of the differences in heap memory vs process memory - 

https://stackoverflow.com/questions/8037920/tomcat-memory-consumption-is-more-than-heap-permgen-spac...

https://www.digitalocean.com/community/tutorials/java-heap-space-vs-stack-memory

[naveench]

Thanks @richard_guerra , thats good observation. We will get the heap and thread dumps and will review them.

[PierreGutierrez · Accepted solution]

Hi @naveench 
I agree with what was mentioned by @richard_guerra 

In the link shared by Richard, the case is similar to the one mentioned.
And it is based on the concept that "Memory usage" is not only Heap Memory, but there are also many more factors:

I also mention that seeing that your process is using 5.3 GB of which you assign a maximum of 4 GB for only heap Memory, makes me think that you can optimize that parameter.  I found some important information about it here: 
Java Experto: rendimiento (upc.edu)   , https://docs.oracle.com/en/graalvm/jdk/21/docs/reference-manual/native-image/optimizations-and-perfo... .

The best way is to apply parameters and apply load tests on your service, to arrive at the parameters that are useful to you. 

I hope it's helpful 💪

[erh_inetum · Accepted solution]

Hi @naveench ,

Completing Richard and Pierre comments, a best practice is configuring the same value for -Xms and -Xmx parameters. And these values shouldn't exceed 4096M. JVMs don´t usually handle propertly values greater than 4GB.
You can also add to the JVM configuration parameters the GC algorithm (G1, Epsilon or Serial) to managed heap. This usually works fine.

Hope it helps.
Regards,
Elena.

[naveench]

thanks for your suggestions @erh_inetum  we will update the configuration parameters.