thread scheduling priority

[Daniel Eischen]

On Mon, 7 Jul 2003, Petri Helenius wrote:
> Daniel Eischen wrote:
> 
> >First, you should say what threading library you are using.
> >I'll answer assuming you are using libkse (if you aren't, ignore me).
> >  
> >
> I'll blame the heat for not mentioning that :-) Yes,
> I'm using libkse.
> 
> >Try forcing libkse to use one KSE.  By default, libkse will
> >create as many KSEs as CPUs that you have.  It is possible
> >
> Actually it seems that it creates double the amount, because of
> the HypeThreading junk, although I have the logical secondary
> cores halted. I still believe it's a mistake not to allow them to
> "go away" altogether but have them configured confusing things
> like system utilities and in this case, libkse;
> # sysctl kern.threads.virtual_cpu=1
> kern.threads.virtual_cpu: 4 -> 1
> 
> >that there is a problem with trying to keep both KSEs active
> >when you only have 2 threads and one or more of them block
> >or sleep.  To do this, set kern.threads.debug=1 and
> >kern.threads.virtual_cpu=1:
> >
> >	# sysctl kern.threads.debug=1
> >	# sysctl kern.threads.virtual_cpu=1
> >
> >Also, the default scheduling policy is SCHED_RR and the RR
> >interval is 20msec.  If your main thread is in a busy loop,
> >no other threads will run for 20 or so msec (assuming 1 KSE).
> >  
> >
> Is there a way to hand over the mutex to the other thread when it's unlocked
> by the main thread? pthread_yield? Is that a no-op if there is no other 
> runnable
> thread? I'd rather have both threads running while they can, using both 
> CPUs.

Mutexes are implemented using direct handoff (I think that's the
term).  Once a mutex is released, it is given to the next thread
that is waiting on it and that thread is then marked runnable.
Idle KSEs are suppose to be woken whenever there are runnable
threads.

For now, just see if things work better with 1 CPU defined.
If not, then all this talk about multiple KSEs isn't your
problem; it is something else.

> I assume the "correct" setting for virtual_cpu in my case would be 2 ?

It shouldn't matter.  Libkse should do the right thing regardless
of how many CPUs you have.  In your case, there may be extra
overhead with having only 2 threads.  Both idling of a KSE and
waking a KSE are system calls, whereas normal thread switches
within the same KSE are not.  Depending on how often your
threads run and block, running those 2 threads on 2 KSEs may
not be as efficient as running them with only 1 KSE.

The way to set the number of KSEs allocated to run scope
process threads is to use pthread_setconcurrency().  This
currently can't be used to reduce the number of KSEs, but
it is on our TODO list.  Also, there is currently no way
to tell libkse not to automatically create as many KSEs
as defined by kern.threads.virtual_cpus.  Perhaps we'll
add an environment variable to do this.

Perhaps we also need a better method of auto-tuning the
number of KSEs.  If KSEs are not utilizing much of their
quantum and there is a lot of idling and waking of them,
then we're better off with less of them.

-- 
Dan Eischen