NFS server bottlenecks

[Nikolay Denev]

On Oct 23, 2012, at 2:36 AM, Rick Macklem <rmacklem at uoguelph.ca> wrote:

> Ivan Voras wrote:
>> On 20 October 2012 13:42, Nikolay Denev <ndenev at gmail.com> wrote:
>> 
>>> Here are the results from testing both patches :
>>> http://home.totalterror.net/freebsd/nfstest/results.html
>>> Both tests ran for about 14 hours ( a bit too much, but I wanted to
>>> compare different zfs recordsize settings ),
>>> and were done first after a fresh reboot.
>>> The only noticeable difference seems to be much more context
>>> switches with Ivan's patch.
>> 
>> Thank you very much for your extensive testing!
>> 
>> I don't know how to interpret the rise in context switches; as this is
>> kernel code, I'd expect no context switches. I hope someone else can
>> explain.
>> 
>> But, you have also shown that my patch doesn't do any better than
>> Rick's even on a fairly large configuration, so I don't think there's
>> value in adding the extra complexity, and Rick knows NFS much better
>> than I do.
>> 
>> But there are a few things other than that I'm interested in: like why
>> does your load average spike almost to 20-ties, and how come that with
>> 24 drives in RAID-10 you only push through 600 MBit/s through the 10
>> GBit/s Ethernet. Have you tested your drive setup locally (AESNI
>> shouldn't be a bottleneck, you should be able to encrypt well into
>> Gbyte/s range) and the network?
>> 
>> If you have the time, could you repeat the tests but with a recent
>> Samba server and a CIFS mount on the client side? This is probably not
>> important, but I'm just curious of how would it perform on your
>> machine.
> 
> Oh, I realized that, if you are testing 9/stable (and not head), that
> you won't have r227809. Without that, all reads on a given file will
> be serialized, because the server will acquire an exclusive lock on
> the vnode.
> 
> The patch for r227809 in head is at:
>  http://people.freebsd.org/~rmacklem/lkshared.patch
> This should apply fine to a 9 system (but not 8.n), I think.
> 
> Good luck with it and have fun, rick
> 
>> _______________________________________________
>> freebsd-hackers at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
>> To unsubscribe, send any mail to
>> "freebsd-hackers-unsubscribe at freebsd.org"

Thanks, I've applied the patch by hand because of some differences and I'm now rebuilding.

In case they are still needed here are the "dd" tests with loopback UDP mount :

http://home.totalterror.net/freebsd/nfstest/udp-dd.html

Over udp writing degrades much worse...