ule+smp: small optimization for turnstile priority lending

[Andre Oppermann]

On 07.11.2012 08:45, David Xu wrote:
> On 2012/11/07 14:17, Jeff Roberson wrote:
>> On Wed, 7 Nov 2012, David Xu wrote:
>>
>>> On 2012/11/06 19:03, Attilio Rao wrote:
>>>> On 9/20/12, David Xu <davidxu at freebsd.org> wrote:
>>>>> I found another scenario in taskqueue, in the function
>>>>> taskqueue_terminate, current thread tries to wake
>>>>> another thread up and sleep immediately, the tq_mutex sometimes
>>>>> is a spinlock. So if you remove one thread load from current cpu
>>>>> before wakeup, the resumed thread may be put on same cpu,
>>>>> so it will optimize the cpu scheduling too.
>>>>
>>>> I think that in order to fit with sched_add() modifies I have in mind
>>>> (see other patches within this thread) wakeup() should grow a new
>>>> argument, or maybe we can use wakeup_flags() new KPI.
>>>> If the latter is the case, I would also propose to let wakeup_one() to
>>>> be absorbed into wakeup_flags() with its own flag.
>>>>
>>>
>>> Yes, I like the idea.
>>
>>> From ~2007
>>
>> http://people.freebsd.org/~jeff/wakeupflags.diff
>>
>> This has some different optimizations that can be done when you have a
>> hinted wakeup.
>>
>
> wakeup_flags is a good start point.
> But what flags should be supported? I think WAKEUP_WILLSLEEP may be
> needed if the current thread will switch out as soon as possible.

WAKEUP_ONE is important for accept sockets.  We don't want all idle
threads to compete and contend on a new connection.

-- 
Andre

> I see you have added WAKEUP_LOCAL and WAKEUP_TAIL. Are they for cache
> optimization ? both of them are arguable.
>
> WAKEUP_LOCAL increases cpu migration, not good, since one benefit of
> ULE is keeping thread on its old cpu, the WAKEUP_LOCAL violates the
> design.
> WAKEUP_LOCAL used in pipe code may not be correct if the pipe only
> has few of bytes to be transfered or receiver only eats a few bytes
> each time, so why bother to move other threads to local cpu ?
> If the other threads have many bytes cached in their old cpu, this
> migration is expensive.
>
> WAKEUP_TAIL is a good idea, I guess that you want to wake up hot-thread
> its code and data are still in its old cpu's cache. But this will lead
> to unfair. I'd like the sleep queue to implement a policy like I did
> in libthr, it picks a thread at tail of queue in a fixed percentage, it
> does have some level of unfair but not fatal at all.
>
>> Thanks,
>> Jeff
>
> _______________________________________________
> freebsd-hackers at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe at freebsd.org"
>
>