cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Too many open files exception

oleksandr_mishc
Advisor

Hello,

Myabe someone clear the issue below:

Began to appear errors on Dynatrace 6.5 server log:

2017-10-11 10:53:38 WARNING [ServerConnector] : org.eclipse.jetty.util.log.JavaUtilLog warn:71
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)

Let see the file handles:

cat /proc/sys/fs/file-nr

4128 0 13069112

And the user limits:

cat /etc/security/limits.conf

#<domain> <type> <item> <value>

* - nofile 2048

dynatrace soft nofile 16000

dynatrace hard nofile 16000

But it's not clear for me because the limit was not reached:

lsof | grep dynatrace | wc -l

1822

After change limits to 18000 and reboot the server all is ok. But why the count was less than set.

6 REPLIES 6

oleksandr_mishc
Advisor

My little mistake. The lsof was run from notprivileged user.

From root:

lsof | grep dynatrace | wc -l
208305

Now the another question, why 208305 if 16000 is limit?

ignacio
Inactive

@Oleksandr M.Having the same issue with Dynatrace Managed. Could you resolve the problem? What did you do to fix it?

Thanks!


Hello @Ignacio Norris P.

Change the limit in the /etc/security/limits.conf like below.

Regards,

Babar


Hello @Babar Q.

I will do that, thanks! I also have to modify /etc/security/limits.conf in the security gateway as well, right?


Hello @Ignacio Norris P.

Yes and below are the steps.

To configure the limit on number of open files:


  1. Create the file /etc/security/limits.d/80-dynatrace.conf with the following content: root hard nofile 500000.
  2. Log out, log back in, and restart Security Gateway.

Make sure the system limit on the number of open files is now higher. To view the system limit, execute the following command:
cat /proc/sys/fs/file-max.

If required, follow the steps below to change the system limit:


  1. Add fs.file-max=1000000 into /etc/sysctl.d/80-dynatrace.conf.
  2. Execute the sysctl -p /etc/sysctl.d/80-dynatrace.conf to apply the new value.

Regards,

Babar


Thank you very much!

Best regards,

Ignacio