mpslsi0 : Trying sleep, but thread marked as sleeping prohibited

[Desai, Kashyap]

> -----Original Message-----
> From: John Baldwin [mailto:jhb at freebsd.org]
> Sent: Thursday, February 23, 2012 8:28 PM
> To: freebsd-stable at freebsd.org
> Cc: Desai, Kashyap; Konstantin Belousov; freebsd-scsi at freebsd.org;
> Kenneth D. Merry; Justin T. Gibbs; McConnell, Stephen
> Subject: Re: mpslsi0 : Trying sleep, but thread marked as sleeping
> prohibited
> 
> On Thursday, February 23, 2012 8:22:07 am Desai, Kashyap wrote:
> >
> > > -----Original Message-----
> > > From: Konstantin Belousov [mailto:kostikbel at gmail.com]
> > > Sent: Thursday, February 23, 2012 2:55 PM
> > > To: Desai, Kashyap
> > > Cc: freebsd-scsi at freebsd.org; freebsd-stable; Justin T. Gibbs;
> Kenneth
> > > D. Merry; McConnell, Stephen
> > > Subject: Re: mpslsi0 : Trying sleep, but thread marked as sleeping
> > > prohibited
> > >
> > > On Thu, Feb 23, 2012 at 05:52:12AM +0530, Desai, Kashyap wrote:
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Konstantin Belousov [mailto:kostikbel at gmail.com]
> > > > > Sent: Thursday, February 23, 2012 12:45 AM
> > > > > To: Desai, Kashyap
> > > > > Cc: freebsd-scsi at freebsd.org; freebsd-stable; Justin T. Gibbs;
> > > > > Kenneth D. Merry; McConnell, Stephen
> > > > > Subject: Re: mpslsi0 : Trying sleep, but thread marked as
> sleeping
> > > > > prohibited
> > > > >
> > > > > On Wed, Feb 22, 2012 at 07:36:42PM +0530, Desai, Kashyap wrote:
> > > > > > Hi,
> > > > > >
> > > > > > I am doing some code changes in mps dirver. While working on
> those
> > > > > changes, I come to know about something which is new to me.
> > > > > > Some expert help is required to clarify my doubt.
> > > > > >
> > > > > > 1. When any irq is register with FreeBSD OS, it sets "
> > > TDP_NOSLEEPING"
> > > > > > pflag. It means though irq in freebsd is treated as thread, We
> > > > > > cannot
> > > > > sleep in IRQ because of " "TDP_NOSLEEPING " set.
> > > > > > 2. In mps driver we have below code snippet in ISR routine.
> > > > > >
> > > > > >
> > > > > >     mps_dprint(sc, MPS_TRACE, "%s\n", __func__);
> > > > > >     mps_lock(sc);
> > > > > >     mps_intr_locked(data);
> > > > > >     mps_unlock(sc);
> > > > > >
> > > > > > I wonder why there is no issue with above code ? Theoretical
> we
> > > > > > cannot sleep in ISR. (as explained in #1) Any thoughts ?
> > > > > >
> > > > > >
> > > > > > 3. I recently added few place msleep() instead of DELAY in ISR
> > > > > > context and I see " Trying sleep, but thread marked as
> sleeping
> > > prohibited".
> > > > > >
> > > > > FreeBSD has several basic ways to prevent a thread from
> executing on
> > > > > CPU.
> > > > > They mostly fall into two categories: bounded sleep, sometimes
> > > > > called blocking, and unbounded sleep, usually abbreviated as
> sleep.
> > > > > The bounded there refers to amount of code executed by other
> thread
> > > > > that hold resource preventing blocked thread from making a
> progress.
> > > > >
> > > > > Examples of the blocking primitives are mutexes, rw locks and rm
> > > locks.
> > > > > The blocking is not counted as sleeping, so interrupt threads,
> which
> > > > > are designated as non-sleeping, still can lock mutexes.
> > > > Thanks for the tech help.  .
> > > >
> > > > As per you comment, So now I understood as "TDP_NOSLEEPING" is
> only
> > > > for unbounded sleep restriction. Just curious to know, What is a
> > > > reason that thread can do blocking sleep but can't do unbounded
> sleep
> > > > ? Since technically we introduced sleeping restriction on
> interrupt
> > > > thread is to avoid starvation and that can be fit with either of
> the
> > > > sleep type. Is this not true ?
> > > No, not to avoid starvation.
> > >
> > > The intent of the blocking primitives is to acquire resources for
> > > limited amount of time. In other words, you never take a mutex for
> > > undefinitely long computation process. On the other hand, msleep
> sleep
> > > usually has no limitations.
> >
> > I got same reply from Ed Schouten. I agree and understood your note.
> Thanks
> for poring knowledge on this area.
> > _but_ only query is when thread take mutex, we don't know when it will
> release. So holding time of mutex is really not known.
> > In case of some bad code, where thread took mutex and not release
> within
> short time. This can eventually match upto msleep restriction as well.
> > Do we have  any checks that thread took long time holding mutext ?
> Similar
> to linux where spinlock has been not release in some specific time, they
> dump
> warnings with backtrace.
> 
> We don't allow code to do unbounded sleeps while holding mutexes either,
> and
> WITNESS warns about doing so.  That ensures that barring an infinite
> loop-type
> bug, mutexes should be held for a bounded amount of time.
Thanks. I understood this concept now. Really helpful conversation.

> 
> --
> John Baldwin