mpslsi0 : Trying sleep, but thread marked as sleeping prohibited

[John Baldwin]

On Thursday, February 23, 2012 8:22:07 am Desai, Kashyap wrote:
> 
> > -----Original Message-----
> > From: Konstantin Belousov [mailto:kostikbel at gmail.com]
> > Sent: Thursday, February 23, 2012 2:55 PM
> > To: Desai, Kashyap
> > Cc: freebsd-scsi at freebsd.org; freebsd-stable; Justin T. Gibbs; Kenneth
> > D. Merry; McConnell, Stephen
> > Subject: Re: mpslsi0 : Trying sleep, but thread marked as sleeping
> > prohibited
> > 
> > On Thu, Feb 23, 2012 at 05:52:12AM +0530, Desai, Kashyap wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: Konstantin Belousov [mailto:kostikbel at gmail.com]
> > > > Sent: Thursday, February 23, 2012 12:45 AM
> > > > To: Desai, Kashyap
> > > > Cc: freebsd-scsi at freebsd.org; freebsd-stable; Justin T. Gibbs;
> > > > Kenneth D. Merry; McConnell, Stephen
> > > > Subject: Re: mpslsi0 : Trying sleep, but thread marked as sleeping
> > > > prohibited
> > > >
> > > > On Wed, Feb 22, 2012 at 07:36:42PM +0530, Desai, Kashyap wrote:
> > > > > Hi,
> > > > >
> > > > > I am doing some code changes in mps dirver. While working on those
> > > > changes, I come to know about something which is new to me.
> > > > > Some expert help is required to clarify my doubt.
> > > > >
> > > > > 1. When any irq is register with FreeBSD OS, it sets "
> > TDP_NOSLEEPING"
> > > > > pflag. It means though irq in freebsd is treated as thread, We
> > > > > cannot
> > > > sleep in IRQ because of " "TDP_NOSLEEPING " set.
> > > > > 2. In mps driver we have below code snippet in ISR routine.
> > > > >
> > > > >
> > > > >     mps_dprint(sc, MPS_TRACE, "%s\n", __func__);
> > > > >     mps_lock(sc);
> > > > >     mps_intr_locked(data);
> > > > >     mps_unlock(sc);
> > > > >
> > > > > I wonder why there is no issue with above code ? Theoretical we
> > > > > cannot sleep in ISR. (as explained in #1) Any thoughts ?
> > > > >
> > > > >
> > > > > 3. I recently added few place msleep() instead of DELAY in ISR
> > > > > context and I see " Trying sleep, but thread marked as sleeping
> > prohibited".
> > > > >
> > > > FreeBSD has several basic ways to prevent a thread from executing on
> > > > CPU.
> > > > They mostly fall into two categories: bounded sleep, sometimes
> > > > called blocking, and unbounded sleep, usually abbreviated as sleep.
> > > > The bounded there refers to amount of code executed by other thread
> > > > that hold resource preventing blocked thread from making a progress.
> > > >
> > > > Examples of the blocking primitives are mutexes, rw locks and rm
> > locks.
> > > > The blocking is not counted as sleeping, so interrupt threads, which
> > > > are designated as non-sleeping, still can lock mutexes.
> > > Thanks for the tech help.  .
> > >
> > > As per you comment, So now I understood as "TDP_NOSLEEPING" is only
> > > for unbounded sleep restriction. Just curious to know, What is a
> > > reason that thread can do blocking sleep but can't do unbounded sleep
> > > ? Since technically we introduced sleeping restriction on interrupt
> > > thread is to avoid starvation and that can be fit with either of the
> > > sleep type. Is this not true ?
> > No, not to avoid starvation.
> > 
> > The intent of the blocking primitives is to acquire resources for
> > limited amount of time. In other words, you never take a mutex for
> > undefinitely long computation process. On the other hand, msleep sleep
> > usually has no limitations.
> 
> I got same reply from Ed Schouten. I agree and understood your note. Thanks 
for poring knowledge on this area.
> _but_ only query is when thread take mutex, we don't know when it will 
release. So holding time of mutex is really not known.
> In case of some bad code, where thread took mutex and not release within 
short time. This can eventually match upto msleep restriction as well.
> Do we have  any checks that thread took long time holding mutext ? Similar 
to linux where spinlock has been not release in some specific time, they dump 
warnings with backtrace.

We don't allow code to do unbounded sleeps while holding mutexes either, and
WITNESS warns about doing so.  That ensures that barring an infinite loop-type
bug, mutexes should be held for a bounded amount of time.

-- 
John Baldwin