SMP question w.r.t. reading kernel variables

[Rick Macklem]

> [good stuff snipped for brevity]
> >
> > 1. Set MNTK_UNMOUNTF
> > 2. Acquire a standard FreeBSD mutex "m".
> > 3. Update some data structures.
> > 4. Release mutex "m".
> >
> > Then, other threads that acquire "m" after step 4 has occurred will
> > see
> > MNTK_UNMOUNTF as set. But, other threads that beat thread X to step
> > 2
> > may
> > or may not see MNTK_UNMOUNTF as set.
> >
> First off, Alan, thanks for the great explanation. I think it would be
> nice if this was captured somewhere in the docs, if it isn't already
> there somewhere (I couldn't spot it, but that doesn't mean
> anything:-).
> 
> > The question that I have about your specific scenario is concerned
> > with
> > VOP_SYNC(). Do you care if another thread performing nfscl_getcl()
> > after
> > thread X has performed VOP_SYNC() doesn't see MNTK_UNMOUNTF as set?
> 
> Well, no and yes. It doesn't matter if it doesn't see it after thread
> X
> performed nfs_sync(), but it does matter that the threads calling
> nfscl_getcl()
> see it before they compete with thread X for the sleep lock.
> 
> > Another
> > relevant question is "Does VOP_SYNC() acquire and release the same
> > mutex as
> > nfscl_umount() and nfscl_getcl()?"
> >
> No. So, to get this to work correctly it sounds like I have to do one
> of the following:
> 1 - mtx_lock(m); mtx_unlock(m); in nfs_sync(), where "m" is the mutex
> used
> by nfscl_getcl() for the NFS open/lock state.
> or
> 2 - mtx_lock(m); mtx_unlock(m); mtx_lock(m); before the point where I
> care
> that the threads executing nfscl_getcl() see MNTK_UMOUNTF set in
> nfscl_umount().
> or
> 3 - mtx_lock(m2); mtx_unlock(m2); in nfscl_getcl(), where m2 is the
> mutex used
> by thread X when setting MNTK_UMOUNTF, before mtx_lock(m); and then
> testing
> MNTK_UMOUNTF plus acquiring the sleep lock. (By doing it before, I can
> avoid
> any LOR issue and do an msleep() without worrying about having two
> mutex locks.)
> 
> I think #3 reads the best, so I'll probably do that one.
> 
> One more question, if you don't mind.
> 
> Is step 3 in your explanation necessary for this to work? If it is, I
> can just create
> some global variable that I assign a value to between mtx_lock(m2);
> mtx_unlock(m2);
> but it won't be used for anything, so I thought I'd check if it is
> necessary?
> 
Oops, I screwed up this question. For my #3, all that needs to be done
in nfscl_getcl() before I care if it sees MNTK_UMOUNTF set is mtx_lock(m2);
since that has already gone through your steps 1-4.

The question w.r.t. do you really need your step 3 would apply to the
cases where I was using "m" (the mutex nfscl_umount() and nfscl_getcl()
already use instead of the one used by thread X).

rick