Major issues with nfsv4

[Konstantin Belousov]

On Sat, Dec 12, 2020 at 03:40:55AM +0000, Rick Macklem wrote:
> J David wrote:
> >On Fri, Dec 11, 2020 at 6:28 PM Rick Macklem <rmacklem at uoguelph.ca> wrote:
> >> I am afraid I know nothing about nullfs and jails. I suspect it will be
> >> something related to when file descriptors in the NFS client mount
> >> get closed.
> >
> >What does NFSv4 do differently than NFSv3 that might upset a low-level
> >consumer like nullfs?
> The opens for one. When a file is opened it finds its way to VOP_OPEN().
> --> For NFSv3 all it does is some client side cache consistency checks.
> --> For NFSv4, it must acquire or update a NFSv4 Open, which is a form
>        of lock that is acquired/updated by an Open operation in an RPC.
>        Then the client stores this locking info in a structure in a linked list
>         off of the mount point.
>        Once all file descriptors for the vnode are closed, then, and only
>        then can a Close operation be done against the server and the linked
>        list data structure be free'd.
>        --> Does having nullfs between the file descriptors and the NFS vnodes
>               for the same file affect when the v_usecount decrements to 0 on
>               the NFS vnode?
>               I don't know. but if it delays it, then these linked list structures
>               will not be free'd as soon and might accumulate.
>               --> The more structures the longer the linked list and the more
>                      overhead/cpu will be used prcessing them.
>        The fact that processes are spending a long time in exit() might
>        be a hint that there are a large # of these NFSv4 Opens to deal with
>        when files are being closed implicitly during exit.
> 
>        As I mentioned, "nfsstat -c -E" will tell you how many Opens there
>        are under the "OpenOwners ..." line.
Nullfs vnodes keep a reference on the lower vnode.  When nullfs vnode
caching is enabled, nullfs vnodes survive after a vfs syscall is finished.

NFSv4 mount automatically sets flag MNTK_NULL_NOCACHE that disables nullfs
vnode cache.