ZFS (or something) is absurdly slow

[Thomas Steen Rasmussen]

On 23-11-2013 09:26, Eitan Adler wrote:
> Every so often I see absurdly slow tasks stuck on "zio->io_cv".
>
> For example a recent "git checkout file.c" did not complete for many minutes
> load: 0.65  cmd: git 74577 [zio->io_cv] 435.58r 0.20u 2.54s 0% 71488k
>
> I have seen "ls ~" take tends of minutes to complete.  Even "ls
> /var/empty" can take just as long.  This length of time is variable
> but is usually much longer than expected.
>
> Does anyone have any suggestions for helping to figure out what is
> taking a long time?
Hello,

If "top -m io -o total" doesn't reveal what is using the disks,
I've had good experiences with the following dtrace script,
you'd need to build dtrace support in your kernel though:

vfsstat.d https://forums.freebsd.org/showpost.php?p=182070&postcount=6

You can also check systat -iostast 1 and check the TPS count for
the disks. The regular (spinning) harddisk can manage 200-300 iops
if it is a regular consumer class disk. Are you "running out" of
iops for some reason ?

Good luck with it,


Best regards
Thomas Steen Rasmussen