Re: panic: nfsv4root ref cnt cpuid = 1

On Sun, Sep 22, 2024 at 7:22 PM Rick Macklem <rick.macklem@gmail.com> wrote:
> I think I know what causes the crashes. The attached trivial patch should
> work around them, but if you cannot apply a source kernel patch, the
> only workaround would be to get rid of "oneopenown".

Unfortunately, FreeBSD NFS is unusable for us without oneopenown due
to the issues with CPU load once enough Opens pile up. (I think our
issues may be at least part of the reason you added the line about
shared libraries to the man page.)

> If you cannot get rid of the "oneopenown" or apply the kernel source patch,
> getting rid of the nullfs mount or enabling delegations might also work around
> this.

We're not using delegations because of, "This can only be enabled when
the file systems being exported to NFSv4 clients are not being
accessed locally on the server." The filesystem is exported read only;
updates are infrequent but must be atomic, so they are performed via
ZFS snapshot receive on the server.

There are two reasons we have been using nullfs. Both are related to
our use case: large number of jobs (jails) where this filesystem must
appear.

The first and simpler issue is reserved port exhaustion. However, the
filesystem is read-only and we've made a change on our end to make it
the only filesystem exported from that server. So it should be
perfectly safe to mount noresvport; that is a very solvable problem
with a few changes to our code.

The second issue is filesystem caching. With one NFS mount the client
machine can (theoretically) keep one copy of the most frequently hit
content from this in cache for everybody. If that's the behavior with
nullfs but not with separate mounts, switching could lead to an
increase in RAM usage or network traffic enough to adversely affect
performance.

What do you think? If an NFS filesystem needs to be mounted in 300-500
different places, is it better to NFS mount it once and nullfs it 500
times or better to NFS mount it 500 times? You certainly know vastly
more about the likely implications than I do.

I will find a way to try the patch.

Thanks!