Machine stops for some seconds with ZFS

[Tommi Lätti]

> After a long time, I've switched back to ZFS on my desktop. It runs
> 8-STABLE/amd64 with two SATA disks and an USB pendrive.
> One-one partition is used from each disk for the zpool, which is encrypted
> using GELI, and the pendrive is there for L2ARC:
>   NAME            STATE     READ WRITE CKSUM
>   data            ONLINE       0     0     0
>     mirror        ONLINE       0     0     0
>       ad0s1d.eli  ONLINE       0     0     0
>       ad1s1d.eli  ONLINE       0     0     0
>   cache
>     da0           ONLINE       0     0     0
>
> Today, after 12 days of uptime the machine has frozen. I could ping it from
> a different machine, even could open a telnet to its ssh port, but I
> couldn't get the ssh banner.
>
> Now I'm building a 9-CURRENT kernel and world to see whether the same
> problem persists with that, and during the make process I've noticed a
> strange thing.
> I build with -j4 (the machine has one dual core CPU), so the fans are
> screaming during the process. But every few minutes (I couldn't recognize
> any patterns in it) the machine goes completely silent (even more silent
> than normally), and everything halts.
>  PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
> 16304 root          1  44    0 37944K  4576K zio->i  1   0:00  0.00% sshd
> 16405 bra           1  44    0 37944K  5012K zio->i  0   0:00  0.00% sshd
> 1064 postfix       1  44    0  9104K  1772K zio->i  1   0:00  0.00% pickup

This sounds like you're being hit by the same performance slowdown
(extensively documented) that seems to affect everybody currently
(maybe not those guys with ssd's or 15k rpm drives in big arrays).
There's a long thread on -STABLE.

Basically how I see it it's impossible to read and write from zfs pool
at the same time which might be caused how the arc cache behaves under
freebsd (didn't have these problems when zfs was still 'unstable'). I
couldn't even watch a 720p video while having small writes (less than
1k every few seconds) to the same array without the smb process going
to zio->i state which seems to indicate a complete block on any i/o.
Combine with 5400 rpm consumer drives... well...

-> switched to opensolaris, performance is now great...

-- 
br,
Tommi