Understanding the FreeBSD locking mechanism

[Yubin Ruan]

On 2017/4/10 0:24, Ryan Stone wrote:
>
>
> On Sun, Apr 9, 2017 at 6:13 AM, Yubin Ruan <ablacktshirt at gmail.com
> <mailto:ablacktshirt at gmail.com>> wrote:
>
>
>     #######1, spinlock used in an interrupt handler
>     If a thread A holding a spinlock T get interrupted and the interrupt
>     handler responsible for this interrupt try to acquire T, then we have
>     deadlock, because A would never have a chance to run before the
>     interrupt handler return, and the interrupt handler, unfortunately,
>     will continue to spin ... so in this situation, one has to disable
>     interrupt before spinning.
>
>     As far as I know, in Linux, they provide two kinds of spinlocks:
>
>       spin_lock(..);   /* spinlock that does not disable interrupts */
>       spin_lock_irqsave(...); /* spinlock that disable local interrupt *
>
>
> In the FreeBSD locking style, a spinlock is only used in the case where
> one needs to synchronize with an interrupt handler.  This is why spinlocks
> always disable local interrupts in FreeBSD.
>
> FreeBSD's lock for the first case is the MTX_DEF mutex, which is
> adaptively-spinning blocking mutex implementation.  In short, the MTX_DEF
> mutex will spin waiting for the lock if the owner is running, but will
> block if the owner is deschedules.  This prevents expensive trips through
> the scheduler for the common case where the mutex is only held for short
> periods, without wasting CPU cycles spinning in cases where the owner thread
> is descheduled and therefore will not be completing soon.

Great explanation! I read the man page at:

 > 
https://www.freebsd.org/cgi/man.cgi?query=mutex&sektion=9&apropos=0&manpath=FreeBSD+11.0-RELEASE+and+Ports

and now clear about MTX_DEF and MTX_SPIN mutexs. But, still a few more
question, if you don't mind:

Is it true that a thread holding a MTX_DEF mutex can be descheduled?
(shouldn't it disable interrupt like a MTX_SPIN mutex?) It is said on
the main page that MTX_DEF mutex is used by default in FreeBSD, so its
usecase must be very common. If a thread holding a MTX_DEF mutex can be 
descheduled, which means that it did not disable interrupt, then we may
have lots of deadlock here, right?

>
>     #######2, priority inversion problem
>     If thread B with a higher priority get in and try to acquire the lock
>     that thread A currently holds, then thread B would spin, while at the
>     same time thread A has no chance to run because it has lower priority,
>     thus not being able to release the lock.
>     (I haven't investigate enough into the source code, so I don't know
>     how FreeBSD and Linux handle this priority inversion problem. Maybe
>     they use priority inheritance or random boosting?)
>
>
> FreeBSD's spin locks prevent priority inversion by preventing the holder
> thread from being descheduled.
>
> MTX_DEF locks implement priority inheritance.

Nice hints. Thanks!

regards,
Yubin Ruan