cvs commit: src/sys/kern kern_mutex.c

[Jeff Roberson]

On Wed, 6 Jun 2007, Bruce Evans wrote:

> On Tue, 5 Jun 2007, John Baldwin wrote:
>
>> On Tuesday 05 June 2007 11:43:03 am Attilio Rao wrote:
>>> 2007/6/5, Attilio Rao <attilio at freebsd.org>:
>>>> 2007/6/5, Bruce Evans <brde at optusnet.com.au>:
>>>>> 
>>>>> I get a "spin lock held too long" panic during (an interrupt in?) acpi
>>>>> initialization on booting non-PREEMPTION SCHED_4BSD SMP.  Haven't tried
>>>>> other cases.
>>>> 
>>>> Do you have a backtrace or any other debugging stuffs available?
>
> No, it's on a laptop with no i/o :-).
>
>>> Mmm, I think I got the bug.
>>> basically, in kern_mutex.c::_mtx_unlock_sleep(), in the not-preemptive
>>> case what happens at some point is:
>>> 
>>> td = curthread;
>>> if (td->td_critnest > 0 || td1->td_priority >= td->td_priority)
>>>          return;
>>> ...
>> If this is the old #ifndef PREEMPTION manual preemption stuff, then just
>> remove it.  I've been wanting to axe it for a while, rwlocks don't do the
>> manual preemption either, and if it is getting in the way it's best to just
>> purge it.
>
> Interesting, I've been wanting to do the opposite -- axe the #ifdef
> PREEMPTION in a different place, in pagezero, since non-manual preemption
> doesn't actually work for SCHED_4BSD (it works for SCHED_ULE, but last
> time I checked, SCHED_ULE was 7% slower for my makeworld benchmark
> since it lets CPUs go idle when there is a runnable process in the
> hope of a better CPU to run on becoming available).  My SMP kernel
> that crashes has this ifdef removed.  However, the crash doesn't
> seem to be caused by pgzero.

You should try with kern.sched.pick_pri = 0.  I have changed this to be 
the default recently.  This weakens the preemption and speeds up some 
workloads.

Are you still experiencing a crash with -current sources?

Jeff

>
> Removing the manual preemption stuff in kern_mutex.c wouldn't affect
> pgzero but might affect operation of the !PREEMPTION case for better
> or worse.  I only use !PREEMPTION on SMP.  With only 1 CPU, something
> like PREEMPTION is needed to get interrupts serviced as soon as possible,
> which is the only reason that I want to preempt things, but with > 1
> CPU there is a good chance of a CPU being idle or going near the
> scheduler soon and thus being scheduled to run interrupt handler(s)
> soon.  The chance increases with the number of CPUs.  !PREEMPTION works
> well in practice with only 2 CPUs (no noticeable interrupt latency),
> at least with manual preemption in kern_mutex.c.
>
> Bruce
>