SMP question w.r.t. reading kernel variables

[Rick Macklem]

[good stuff snipped for brevity]
> 
> 1. Set MNTK_UNMOUNTF
> 2. Acquire a standard FreeBSD mutex "m".
> 3. Update some data structures.
> 4. Release mutex "m".
> 
> Then, other threads that acquire "m" after step 4 has occurred will
> see
> MNTK_UNMOUNTF as set. But, other threads that beat thread X to step 2
> may
> or may not see MNTK_UNMOUNTF as set.
> 
First off, Alan, thanks for the great explanation. I think it would be
nice if this was captured somewhere in the docs, if it isn't already
there somewhere (I couldn't spot it, but that doesn't mean anything:-).

> The question that I have about your specific scenario is concerned
> with
> VOP_SYNC(). Do you care if another thread performing nfscl_getcl()
> after
> thread X has performed VOP_SYNC() doesn't see MNTK_UNMOUNTF as set?

Well, no and yes. It doesn't matter if it doesn't see it after thread X
performed nfs_sync(), but it does matter that the threads calling nfscl_getcl()
see it before they compete with thread X for the sleep lock.

> Another
> relevant question is "Does VOP_SYNC() acquire and release the same
> mutex as
> nfscl_umount() and nfscl_getcl()?"
> 
No. So, to get this to work correctly it sounds like I have to do one
of the following:
1 - mtx_lock(m); mtx_unlock(m); in nfs_sync(), where "m" is the mutex used
    by nfscl_getcl() for the NFS open/lock state.
or
2 - mtx_lock(m); mtx_unlock(m); mtx_lock(m); before the point where I care
    that the threads executing nfscl_getcl() see MNTK_UMOUNTF set in nfscl_umount().
or
3 - mtx_lock(m2); mtx_unlock(m2); in nfscl_getcl(), where m2 is the mutex used
    by thread X when setting MNTK_UMOUNTF, before mtx_lock(m); and then testing
    MNTK_UMOUNTF plus acquiring the sleep lock. (By doing it before, I can avoid
    any LOR issue and do an msleep() without worrying about having two mutex locks.)

I think #3 reads the best, so I'll probably do that one.

One more question, if you don't mind.

Is step 3 in your explanation necessary for this to work? If it is, I can just create
some global variable that I assign a value to between mtx_lock(m2); mtx_unlock(m2);
but it won't be used for anything, so I thought I'd check if it is necessary?

Thanks again for the clear explanation, rick