couple of sched_ule issues

[Andriy Gapon]

on 03/11/2011 22:17 Jeff Roberson said the following:
> On Thu, 15 Sep 2011, Andriy Gapon wrote:
> 
>>
>> This is more of a "just for the record" email.
>> I think I've already stated the following observations, but I suspect that they
>> drowned in the noise of a thread in which I mentioned them.
>>
>> 1. Incorrect topology is built for single-package SMP systems.
>> That topology has two levels ("shared nothing" and "shared package") with exactly
>> the same CPU sets.  That doesn't work well with the rebalancing algorithm which
>> assumes that each level is a proper/strict subset of its parent.
>>
>> 2. CPU load comparison algorithms are biased towards lower logical CPU IDs.
>> With all other things being equal the algorithms will always pick a CPU with a
>> lower ID.  This creates certain load asymmetry and predictable patterns in load
>> distribution.
> 
> If all other things truly are equal why does selecting a lower cpu number matter?

Honestly, I am not sure.
This bias could definitely produce an "inequality" from the point of view of how
many threads each CPU gets.  But I can not say if there could be any
"inequality" in how much CPU time each thread gets.  But I don't rule out that
that might be possible...
Please also see this post of mine:
http://thread.gmane.org/gmane.os.freebsd.current/133763/focus=134192

>> Another observation.
>> It seems that ULE makes a decision about thread-to-CPU affinity at the time
>> when a
>> thread gets switched out.  This looks logical from the implementation point of
>> view.  But it doesn't seem logical from a general point of view - when the thread
>> will be becoming running again its affinity profile may become completely
>> different.  I think that it would depend on how much a thread actually spends not
>> running.
> 
> The decision is made at sched_add() time.  sched_pickcpu() does the work and
> selects the run-queue we will be added to.  We consider the CPU that the thread
> was last running on but the decision is made at the time that a run queue must
> be selected.

Ah, yes.  But, OTOH, it doesn't look like sched_add would be called for a
CPU-bound thread (no voluntary sleeps) ?

-- 
Andriy Gapon