Slow disk problems on a monitored host

[matthew_gardner]

Hello Everyone,

I am trying to understand what might be causing some slow disk problems on a monitored host. We can see that latency shoots up in intervals of 5 minutes, and sometimes reaches 300ms+

Just before that, we can see the throughput go up, but not a ton of data is being written:

From the IOPS metrics, we can see after writes/s go up, so does the queue length:

[ChadTurner]

The host in question, is it a SQL host? Are there any commits taking place that might cause the latency? 

[matthew_gardner]

The server uses InterSystems IRIS technology. Which is a database technology, so that could be part of the problem since we see latency go up after some writes.

[AntonioSousa · Accepted solution]

There are a lot of cases where you might see a slow disk problem. Some I have encountered:

• You have a slow HDD disk, with a slow RPM. This doesn't seem to be your case.
• You are using logic volumes, or in some way, the disk is virtualized. Another system is using it extensively, so you're getting high latency in your case.
• You have a filesystem that is clogged. It might be due having a lot of files in a directory, it is heavily fragmented, etc.
• You have a disk with problems. Check the SMART of your disks.

These are some I remember fast. On the funny side, I haven't experienced the following type of proble, at least that I know of:

https://www.youtube.com/watch?v=tDacjrSCeq4