SMPing libpthread

[David Xu]

----- Original Message ----- 
From: "Daniel Eischen" <eischen at pcnet1.pcnet.com>
To: "David Xu" <davidxu at freebsd.org>
Cc: <freebsd-threads at freebsd.org>
Sent: Saturday, April 26, 2003 12:52 PM
Subject: Re: SMPing libpthread


> On Thu, 24 Apr 2003, David Xu wrote:
> 
> > I have put a patch to enable userland support SMP scheduling. 
> > http://people.freebsd.org/~davidxu/libpthread_smp.diff
> > The patch works on my SMP machine, but not fully tested,
> > and I will work on idle kses stuffs. At least, it seems
> > the SMP speed is not slower than UP. :-)
> 
> David, I noticed that we hold the scheduling lock before and
> after calling the scheduler.  Is this necessary?  And if so,
> is it necessary to hold hold it after return from the
> scheduling switch?  One you're in the scheduler, and choose
> another thread (releasing the lock after doing so), shouldn't
> you be able to switch to it without having the lock?
> 

I am trying to do things in atomic way. because I found on SMP,  thread
state is always out of synchronized in current code.  I want to make
state change and context switch in an atomic operation,  when thread is
in state transit,  no other threads can use _thr_setrunnable_unlocked()  etc
to change its state,  this avoids lots of "thread already in priority queue"
messages,  and if  I test the "in priority queue" flag every where, I sometimes
lose chance to wakeup a thread because of some unknown races, whole
process just suddenly stops there.
I am just copying the idea from current kernel code,  I don't intend to inventing
new things. :-)

One deadlock I found is in kse_sched_multi(), there is an optimization:
if (curthread == NULL)
		;  /* Nothing to do here. */
	else if ((curthread->need_switchout == 0) &&
	    (curthread->blocked == 0) && (THR_IN_CRITICAL(curthread))) {
		/*
		 * Resume the thread and tell it to yield when
		 * it leaves the critical region.
		 */

These code cause deadlock,  at least there is a deadlock between malloc/free and
static mutex initializing code,  I think unless we believe that thread is locking a leaf 
node, it is not safe to give cpu to the thread again. I suggest comment out these 
optimization code.
I am still working on it,  hope that I can post a patch soon.
A bad news is kernel seems too unstable when testing KSE program,  it just silently
locks up,  not panic,  no dump,  just locks up, this is newest kernel source code. :(

> -- 
> Dan Eischen
> 
>