Multiplexed threads signals.

[Daniel Eischen]

On Mon, 12 May 2003, Julian Elischer wrote:
> 
> I'm looking at trying to work out who is to deliver a signal in an M:N
> process. We never answered some of these questions.. Altering ht ecode I
> find that I need to answer them now..
> 
> 
> This started off as a comment in the code but it kept growing...
> 
> better to discuss it...
> 
>     /* Here we need to decide our policy
>      * with respect to waking up threads for async signal
>      * delivery. Generally in this library, async signals are not
>      * actually delivered to teh process, but rather a notification is 
>      * delivered during an upcall with a mask similar to that 
>      * used in the sigpending() call, and the library 'fetches' 
>      * the waiting signal using sigwait();

I was think about the _kernel_'s sigwait/sigwaitinfo/sigtimedwaitinfo().
If the UTS allocated a separate thread whose only job was
to receive signals using sig[timed]wait[info]() and dispatch
them to other threads would this make things easier?
This would be only for async signals; sync signals would
still be delivered as they would for a normal thread/process.
Or perhaps some new syscall (_kse_sigtimedwait())...

The only thing that the UTS would need is to be able
to redirect the signal to a thread running or blocked
in the kernel or a scope process thread that has no upcall
stack (assuming we add this optimization at some point).

The only problem is telling the kernel that we actually
found a thread to handle the signal.  Perhaps sigsuspend()
is better to use, and then we can retrieve it using
sigwaitinfo().  The kernel needs to know that the UTS
has a thread for the signal so it can remove it from
the pending signals and also allow a sender using
sigqueue() to wakeup.

> (0) - threads in sigwait for the signal get first crack.
> 
> (1)  * Firstly, if we interrupted a thread in userland,
>      * don't wake up ANY kernel threads, When we finish 
>      * it's going back. It can take the signal with it..
>      * SYNC signals take this path, but do not create upcalls.
>      * Async signals only do this if there is an upcall possible.
> 
> I think that is probably ok..
> Only async calls proceed beyond here.
> 
> (2)  * Secondly we will look for an idle upcall
>      * and make it do an upcall. Better to bring up
>      * another upcall than to interrupt a thread that
>      * may not deserve being interrupted.
> 
> Not really a problem.
>  
> (3)  * If there is a thread runnable that is going
>      * to return to userland upon resumption
>      *  (i.e. been pre-empted). Set its bit and
>      * let it handle it. Maybe give it a boost
>      * if its low priority. If several, choose
>      * highest priority. Don't do this if an upcall
>      * is not possible, for that thread.
>      * An actual upcall thread takes priority.
> 
> Existing code will force this to convert to an upcall
> on transition to userland.
> 
> (4)  * If there is a thread actually running NOW
>      * on another CPU and in userland, whack it
>      * with an IPI and leave the rest to it.
>      * We can not tell if it will create an upcall
>      * on getting whacked since we can't look at it's
>      * upcall mailbox.

Is there a need to whack it, or can we just wait until
it (or another thread) enters the kernel again or gets
swapped out and restarted?

> This results in option (1) for the other thread.
> But if it's in the UTS, no upcall will result.
> If this happens we are left with an un-notified signal, but
> at least we know that there will be at least one thread coming down
> eventually. if a process decides to run in purely non-upcall mode,
> then we can not deliver any async signals to it.. This could be a
> problem. This suggests that we need a way to deliver signals without
> the complicity of the UTS. Maybe if the next upgoing thread
> is not capable of doing an upcall, we just deliver on the stack as per
> normal.

In libpthread, there can always be one KSE with upcalls
if we so choose it.  Even if we are emulating libthr,
I don't see the harm in creating one extra KSE just
for this purpose...

>      * To some extent
>      * this suggests that we look for pending signals
>      * when ENTERING the kernel for a syscall. One could
>      * make the point that it MIGHT have received a clock
>      * interrupt 1 instraction before enterring the syscall,
>      * and then continued on to do the syscall.
>      * This would be akin to returning an ERESTART
>      * but before actually doing the meat of the syscall.
> 
> Peter suggested that we keep a per-KSEGRP mask
> (lazily set) and that if we encounter a ksegrp with a signal
> unmasked and registered as not multiplexing, we just deliver up to it as
> on its stack.. (i.e. system scop threads just get signals as per normal)
> this coincides with what  libthr does in the degenerate case.

Don't forget we still need a way to redirect signals.

-- 
Dan Eischen