Throughput problems with NFS between Linux and FreeBSD

[Richard Sharpe]

On Fri, 19 Sep 2003, John-Mark Gurney wrote:

> Richard Sharpe wrote this message on Fri, Sep 19, 2003 at 10:38 -0700:
> > We recently encountered a problem with NFS throughput between a FreeBSD 
> > server (we are using 4.6.2, but the same code seems to be in 5.1 as well).
> > 
> > When using Linux 2.4.19 or 2.4.21 as a client, although this might extend 
> > to other clients, and copying a large file, you will see the behavior 
> > shown in 
> 
> [...]
> 
> > The problem seems to be the following code
> > 
> >         if (so->so_type == SOCK_STREAM)
> >                 siz = NFS_MAXPACKET + sizeof (u_long);
> >         else
> >                 siz = NFS_MAXPACKET;
> >         error = soreserve(so, siz, siz);
> > 
> > in src/sys/nfs/nfs_syscalls.c.
> > 
> > We added a sysctl to allow finer control over what is passed to soreserve.
> > 
> > With the fix in, it goes up to around wire speed when lots of data is in 
> > the cache.
> 
> What is the fix?  You don't say what adjustments to soreserve's parameters
> are necessary to improve performance?  Have you done testing against other
> clients to see how your changes will affect performance on those machines?

The beest fix is:

         if (so->so_type == SOCK_STREAM)
-                siz = NFS_MAXPACKET + sizeof (u_long);
+                siz = NFS_MAXPACKET + sizeof (u_long) + MSS;
         else
                 siz = NFS_MAXPACKET;
         error = soreserve(so, siz, siz);

in src/sys/nfs/nfs_syscalls.c.

Since the client should only hang onto the ack for one segment, and that 
will work even if you have end-to-end jumbo frames. A simpler fix might be 
to replace MSS with 2048.

Regards
-----
Richard Sharpe, rsharpe[at]ns.aus.com, rsharpe[at]samba.org, 
sharpe[at]ethereal.com, http://www.richardsharpe.com