NFS Client changing it's source address? (FreeBSD 10.2)

[Rick Macklem]

Frank de Bot wrote:
> Hello,
> 
> I have a server running FreeBSD 10.2. It has several NFS mounts.
> Frequently my NFS mount hang (v3). After a little investigation it looks
> like FreeBSD has chosen a wrong source address for it's connections and
> all packets are departing from the wrong interface.
> 
> Sockstat output:
> 
> [root at host004 ~]# sockstat -4 | grep 2049
> root     ssh        14689 3  tcp4   10.4.2.4:20499        10.4.2.5:22
> ?        ?          ?     ?  tcp4   10.13.37.4:672        10.13.37.2:2049
> ?        ?          ?     ?  tcp4   79.x.x.210:905    10.13.37.2:2049
> ?        ?          ?     ?  tcp4   79.x.x.210:992    10.13.37.2:2049
> 
> tcpdump confirms nfs connection are trying to get out via the 79.x.x.x
> interface
> 
> My fstab for the nfs mounts look like:
> 
> 10.13.37.2:/tank/hostingbase      /opt/jails/hostingbase       nfs
> nfsv3,ro,noatime,async,noauto    0       0
> 
> /opt/jails/hostingbase          /opt/jails/test01       nullfs
> ro,noatime,noauto      0       0
> 10.13.37.2:/tank/hosting/test   /opt/jails/test01/opt   nfs
> nfsv3,noatime,async,rw,noauto   0       0
> tmpfs                           /opt/jails/test01/shm   tmpfs
> rw,size=512000000,noauto      0       0
> 
> /opt/jails/hostingbase          /opt/jails/test2       nullfs
> ro,noatime,noauto      0       0
> 10.13.37.2:/tank/hosting/test2   /opt/jails/test2/opt   nfs
> nfsv3,noatime,async,rw,noauto   0       0
> tmpfs                           /opt/jails/test2/shm   tmpfs
> rw,size=512000000,noauto      0       0
> 
> 
> The change of source address looks to be happening after a nfs
> connection  is re-established. At first everything works, I leave the
> server idling (it's a test  server) and after that the mounts are hanging
> 
If the client side of the kernel RPC needs a new TCP connection to the server,
it will do a soconnect(). It will be done by whatever process/thread is doing
the NFS I/O syscall at the time that the connection isn't available.

I'd guess that a process/thread that is running in a jail that can't see the
10.13.37 network does this soconnect() and then it is broken.
I am not conversant w.r.t. jails and I don't know of any clever way around this.

A couple of "hackish" workarounds that might work:
- Reconfigure your NFS server so that it never drops idle TCP connections.
  (FreeBSD never drops an NFS TCP connection until umount, but some others
   like Solaris NFS servers drop idle connections.)
- Make your NFS server accessible through the 79.n.n network instead of 10.13.37.
- Write a daemon that does a stat() of a file on the NFS mount once per minute,
  so the NFS connection is never idle long enough to be disconnected.

rick

> 10.2-RELEASE #0 r286666 is the current running version.
> 
> Regards,
> 
> Frank de Bot
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
>