mpslsi0 : Trying sleep, but thread marked as sleeping prohibited

[Desai, Kashyap]

> -----Original Message-----
> From: Konstantin Belousov [mailto:kostikbel at gmail.com]
> Sent: Thursday, February 23, 2012 2:55 PM
> To: Desai, Kashyap
> Cc: freebsd-scsi at freebsd.org; freebsd-stable; Justin T. Gibbs; Kenneth
> D. Merry; McConnell, Stephen
> Subject: Re: mpslsi0 : Trying sleep, but thread marked as sleeping
> prohibited
> 
> On Thu, Feb 23, 2012 at 05:52:12AM +0530, Desai, Kashyap wrote:
> >
> >
> > > -----Original Message-----
> > > From: Konstantin Belousov [mailto:kostikbel at gmail.com]
> > > Sent: Thursday, February 23, 2012 12:45 AM
> > > To: Desai, Kashyap
> > > Cc: freebsd-scsi at freebsd.org; freebsd-stable; Justin T. Gibbs;
> > > Kenneth D. Merry; McConnell, Stephen
> > > Subject: Re: mpslsi0 : Trying sleep, but thread marked as sleeping
> > > prohibited
> > >
> > > On Wed, Feb 22, 2012 at 07:36:42PM +0530, Desai, Kashyap wrote:
> > > > Hi,
> > > >
> > > > I am doing some code changes in mps dirver. While working on those
> > > changes, I come to know about something which is new to me.
> > > > Some expert help is required to clarify my doubt.
> > > >
> > > > 1. When any irq is register with FreeBSD OS, it sets "
> TDP_NOSLEEPING"
> > > > pflag. It means though irq in freebsd is treated as thread, We
> > > > cannot
> > > sleep in IRQ because of " "TDP_NOSLEEPING " set.
> > > > 2. In mps driver we have below code snippet in ISR routine.
> > > >
> > > >
> > > >     mps_dprint(sc, MPS_TRACE, "%s\n", __func__);
> > > >     mps_lock(sc);
> > > >     mps_intr_locked(data);
> > > >     mps_unlock(sc);
> > > >
> > > > I wonder why there is no issue with above code ? Theoretical we
> > > > cannot sleep in ISR. (as explained in #1) Any thoughts ?
> > > >
> > > >
> > > > 3. I recently added few place msleep() instead of DELAY in ISR
> > > > context and I see " Trying sleep, but thread marked as sleeping
> prohibited".
> > > >
> > > FreeBSD has several basic ways to prevent a thread from executing on
> > > CPU.
> > > They mostly fall into two categories: bounded sleep, sometimes
> > > called blocking, and unbounded sleep, usually abbreviated as sleep.
> > > The bounded there refers to amount of code executed by other thread
> > > that hold resource preventing blocked thread from making a progress.
> > >
> > > Examples of the blocking primitives are mutexes, rw locks and rm
> locks.
> > > The blocking is not counted as sleeping, so interrupt threads, which
> > > are designated as non-sleeping, still can lock mutexes.
> > Thanks for the tech help.  .
> >
> > As per you comment, So now I understood as "TDP_NOSLEEPING" is only
> > for unbounded sleep restriction. Just curious to know, What is a
> > reason that thread can do blocking sleep but can't do unbounded sleep
> > ? Since technically we introduced sleeping restriction on interrupt
> > thread is to avoid starvation and that can be fit with either of the
> > sleep type. Is this not true ?
> No, not to avoid starvation.
> 
> The intent of the blocking primitives is to acquire resources for
> limited amount of time. In other words, you never take a mutex for
> undefinitely long computation process. On the other hand, msleep sleep
> usually has no limitations.

I got same reply from Ed Schouten. I agree and understood your note. Thanks for poring knowledge on this area.
_but_ only query is when thread take mutex, we don't know when it will release. So holding time of mutex is really not known.
In case of some bad code, where thread took mutex and not release within short time. This can eventually match upto msleep restriction as well.
Do we have  any checks that thread took long time holding mutext ? Similar to linux where spinlock has been not release in some specific time, they dump warnings with backtrace.

~ Kashyap
> 
> You do not want the interrupt thread to be put off the processor for
> undefined time, so sleep is prohibited.
> 
> Another issue is that sleeping locks do not do priority propagation to
> the resource owners, while turnstiles used for blocking do. This way,
> interrupt thread waiting for mutex donates its priority to the current
> mutex owner, or at least it shall do.
> 
> >
> > I will be able to progress on my work based on your comment. A much
> thanks for correcting my doubt.
> >
> > ~ Kashyap
> >
> > >
> > > Examples of the sleeping primitives are msleep(), sx locks, lockmgr
> > > locks and conditional variables.
> > >
> > > In essence, the locking facilities are split into several classes
> > > that form the hierarchy, and you cannot legally obtain the lock of
> > > higher class while holding a lock of lower class:
> > > 	spin mutexes -> blocking locks -> sleeping locks.
> > > It establishes some meta-order on the all locks.
> > >
> > > Does this make sense ?
> >