FreeBSD 9.0 hangs on heavy I/O

[Freddie Cash]

On Tue, May 29, 2012 at 2:30 PM, Kees Jan Koster <kjkoster at gmail.com> wrote:
> Dear Freddie,
>
>> Granted, I haven't played with gsched yet (most of our high-I/O
>> systems are ZFS), so there may be a way to use it across-GEOMs.
>
> From my previous experiments ZFS suffers the same fate when there is heavy write activity. Reads just don't get served in time.
>
> How do you deal with that?

We're currently only using FreeBSD (and ZFS) on our backups servers.
The two main servers do rsync backups for ~150 remote Linux servers
and FreeBSD firewalls (1 server does the elementary and secondary
schools; the other server does the admin sites).  Then they do zfs
sends to a third system off-site.

Thus, our workloads tend to be fairly one-sided (all reads on the zfs
send side; all writes on the zfs recv side; mostly reads on the rsync
side side with some writes).  And, most of our working set fits into
ARC/L2ARC.  Cache devices really help, as most reads come from the
L2ARC, while most writes go straight through to the pool.

We're still a year or so away from our ultimate goal of using
FreeBSD+ZFS+NFS to create a separate/proper SAN/NAS tier for our
virtual servers.  At that point, we'll look a little deeper into
things, and experiment with different L2ARC/ZIL setups to optimise
read and write paths.


-- 
Freddie Cash
fjwcash at gmail.com