NFS 4.1 RECLAIM_COMPLETE FS failed error

[Rick Macklem]

Daniel Engel wrote:
>I am setting up an environment with FreeBSD 11.1 sharing a ZFS datastore to vmware >ESXI 6.7.  There were a number of errors with NFS 4.1 sharing that I didn't >understand until I found the following thread.
>
>    <https://lists.freebsd.org/pipermail/freebsd-stable/2018-March/088486.html>
>
>I traced the commits that Rick has made since that thread and merged them 'head' >into 'stable':
>
>    'svnlite checkout http://svn.freebsd.org/base/release/11.1.0/'
>    'svnlite merge -c 332790 http://svn.freebsd.org/base/head'
>    'svnlite merge -c 333508 http://svn.freebsd.org/base/head'
>    'svnlite merge -c 333579 http://svn.freebsd.org/base/head'
>    'svnlite merge -c 333580 http://svn.freebsd.org/base/head'
>    'svnlite merge -c 333592 http://svn.freebsd.org/base/head'
>    'svnlite merge -c 333645 http://svn.freebsd.org/base/head'
>    'svnlite merge -c 333766 http://svn.freebsd.org/base/head'
>    'svnlite merge -c 334396 http://svn.freebsd.org/base/head'
>    'svnlite merge -c 334492 http://svn.freebsd.org/base/head'
>    'svnlite merge -c 327674 http://svn.freebsd.org/base/head'
>
>That completely fixed the connection instability, but the NFS share was still mounting >read-only with a RECLAIM_COMPLETE error.  So, I manually applied the first patch >from the previous thread and everything started working:
>
>    --- fs/nfsserver/nfs_nfsdserv.c.savrecl     2018-02-10 20:34:31.166445000 -0500
>    +++ fs/nfsserver/nfs_nfsdserv.c     2018-02-10 20:36:07.947490000 -0500
>    @@ -4226,10 +4226,9 @@ nfsrvd_reclaimcomplete(struct nfsrv_desc
>           goto nfsmout;
>        }
>        NFSM_DISSECT(tl, uint32_t *, NFSX_UNSIGNED);
>    +   nd->nd_repstat = nfsrv_checkreclaimcomplete(nd);
>        if (*tl == newnfs_true)
>    -           nd->nd_repstat = NFSERR_NOTSUPP;
>    -   else
>    -           nd->nd_repstat = nfsrv_checkreclaimcomplete(nd);
>    +           nd->nd_repstat = 0;
>
>The question is: Did I miss something?  Is there an alternate change already in SVN >that does the same thing better, or is there some corner case preventing this patch >from being finalized that I just haven't run into yet?
Andreas Nagy has been doing quite a bit of testing for me w.r.t the ESXi 6.5
client, but several serious issues (which appear to be violations of the RFC to me)
have not yet been resolved.

This email summarizes then:
http://docs.FreeBSD.org/cgi/mid.cgi?YTOPR0101MB0953E687D013E2E97873061ADD720

He recently reported that 6.7 worked better, but he has not yet sent me any
packet traces, so I don't know which issues still exist for 6.7.
I have committed a few things that didn't break the RFC, such as adding
BindConnectiontoSession, but I haven't committed anything else yet,
due to concerns w.r.t. violating the RFC. (The above email thread discusses that.)

I do plan on doing something once I get packet traces from Andreas, but be
forewarned that VMware states "FreeBSD is not a supported server" and that
is certainly true. Andreas uses connection trunking. You might be ok with a
single TCP connection unless the server reboots.
(He runs a bunch of patches I gave him, some of which definitely violate
 the RFC.)

All I can suggest is that you keep an eye on freebsd-current@ for any email
about commits to handle the ESXi client better.

So, this is very much a work in progress, rick