svn commit: r192535 - head/sys/kern

[John Baldwin]

On Thursday 21 May 2009 11:41:00 am M. Warner Losh wrote:
> In message: <alpine.BSF.2.00.0905211610140.18790 at fledge.watson.org>
>             Robert Watson <rwatson at FreeBSD.org> writes:
> : On Thu, 21 May 2009, John Baldwin wrote:
> : 
> : >>>>   Move the M_WAITOK flag in notify() into an M_NOWAIT one in order to
> : > match
> : >>>>   the behaviour alredy present with the further malloc() call in
> : >>>>   devctl_notify().
> : >>>>   This fixes a bug in the CAM layer where the camisr handler finished 
to
> : >>>>   call camperiphfree() (and subsequently destroy_dev() resulting in a 
new
> : >>>>   dev notify) while the xpt lock is held.
> : >>> This is wrong. You cannot call destroy_dev() while holding any mutex. 
> : >>> Taking this into account, it makes no sense to use M_NOWAIT in 
notify().
> : >>
> : >> As long as devctl_notify() also calls M_NOWAIT and if not available 
skips 
> : >> "silently" it just does the same thing, I think this approach is more 
> : >> consistent.
> : >>
> : >> It remains, though, the fact to fix CAM when calling destroy_dev(). 
Maybe 
> : >> we should add a witness_warn() there?
> : >
> : > I agree with kib, this should be reverted and CAM fixed instead.  I also 
> : > agree that M_NOWAIT use should be limited where possible.
> : 
> : devctl_notify() probably needs to grow a sleepable flag, or perhaps we 
need 
> : two variations, one that can sleep.
> 
> devctl_notify() has expanded well beyond its original needs.  Having
> an extra case for sleeping is the wrong way to solve this problem.
> Really.  We're adding hacks on hacks on hacks here and we need to step
> back and think.
> 
> I specifically didn't put in CDEV notifications into devd when I
> originally did it because one can get the same notification via
> kevents on /dev.  Maybe the right answer is to remove this stuff
> entirely and update devd to do that instead?  It isn't a lot of code,
> and should provide equivalent functionality without needing to change
> the rules of the game when it comes to destroy_dev().  Especially this
> close to the code slush...
> 
> Comments?

destroy_dev() is not a good idea to call with a mutex held period.  
devctl_notify() is the least of one's worries in that case.  In general the 
code holding a mutex over destroy_dev() should be fixed and I think 
devctl_notify() can be left unchanged.  destroy_dev() is a draining operation 
similar to bus_teardown_intr(), callout_drain(), taskqueue_drain(), 
if_detach() (which doesn't drain yet, but needs to), etc.  One simply cannot 
hold locks across those operations.

-- 
John Baldwin