Slow nfsd write performance, tweaks needed

[Mark Schouten]

Hi,


Mark Schouten <mark at tuxis.nl> , 13-11-2014 11:17:
I am in the process of switching from a ZFS On Linux-based NFS-server to a FreeBSD-based NFS-server. The FreeBSD implementation of ZFS is way superiour over ZoL, and the box serves as storage for a virtualizationplatform, so stability is welcome. :) 
 
 
The box is stable, but performs terribly. Surely, I'm doing something wrong, but I would like some tips and tricks to speed things up. 

Thanks for all your responses. I've setup a testserver to do some benchmarks, here are the results. Things are tested from inside a VM, that runs inside a Proxmox-VM, that runs on my Desktop (that performs pretty good, BTW :))


So, command used to benchmark is: 'bonnie++ -u root -d . -r 1024'


Locally on the zfs/nfs/ctl-server itself:
   1m42.91s
iscsi (using the new ctld on FreeBSD 10):
   1m55.203s
nfs on linux desktop (on which the vms are running too)
   4m42.267s
ufs via nfs from FreeBSD 10:
   20m35.644s
zfs via nfs from FreeBSD 10:
   24m23.225s
zfs with sync disabled via nfs from FreeBSD 10:
   21m0.191s


The same FreeBSD testbox is used for all tests, which was doing nothing but the benchmark. What I've learned from these benchmarks is this:
* NFS on FreeBSD just sucks, bigtime
* NFS on Linux performs 'ok'
* iScsi on FreeBSD is fast, very fast


Two questions remain: 
* How stable is ctl(d)?
* How does it perform with more iops and clients?


If anyone has any input on that, I'd be grateful.


Regards, 

-- 
Kerio Operator in de Cloud? https://www.kerioindecloud.nl/
Mark Schouten | Tuxis Internet Engineering
KvK: 61527076 | http://www.tuxis.nl/
T: 0318 200208 | info at tuxis.nl

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4549 bytes
Desc: Electronic Signature S/MIME
URL: <http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20141117/b192baa1/attachment.bin>