[RFC] sema_wait_sig

[Robert Watson]

On Mon, 26 Nov 2012, Jeff Roberson wrote:

> I think that's the underlying issue here.  We don't use semaphores, they 
> aren't implemented well on FreeBSD.  They are there as a token effort at 
> providing an API that we felt was better served in other ways.  If we want 
> to support this API then we should support it well and it should be written 
> to the standards of the other primitives.
>
> Attilio wants to solve the problem by eliminating the API entirely because 
> he knows how long it takes to write a good primitive.  Those who are calling 
> to support it should consider the effort it takes.  It's not free. Calling 
> it supported rather than deprecated means driver authors will expect it to 
> be up to the quality standards of the rest of the kernel. This could have 
> unintended consequences.  Idioms will be developed.  Code will be copied and 
> propagated.  All while using something terribly inefficient.  Users will 
> wonder why their driver performs so much better on Linux.
>
> I am responsible for one use of this code and I only did so knowing that it 
> wasn't for anything performance critical and I was being lazy with a 
> considerable porting effort ahead of me.  I think my approach at this point 
> would be either to entirely eliminate sema or rewrite it appropriately. 
> Adding new consumers and hacking in a new feature is just another half 
> measure.  Which of the other solutions is best is up to individual bias.

I had originally followed up on Jeff's comment privately, but probably would 
have been more useful to do so publically -- I'm a bit late to the discussion, 
but:

FWIW, I quite agree with these points -- I'd hoped sema(9) would go away years 
ago, since its underuse has a number of implications:

(1) Poor integration with debugging support -- e.g., WITNESS, profiling, etc.
     These features make a huge difference in our development model,
     effectively preventing us from shipping many common forms of deadlock in
     the field at all, but come at significant initial cost in developing the
     underlying locking primitives.  sema(9) is an abandoned step child and
     should not persist in its current state.

(2) Poor optimisation -- the great work folk like John and Attilio have done
     on adaptive locking support, etc, massively improved performance in the
     6.x and 7.x timeframe, addressing necessary contention much more
     efficiently.  The reality is that contention occurs frequently because
     communication occurs frequently; obviously, we try to use work
     distribution, improved lock granularity, etc, where we can, but sometimes
     you must communicate and you can't use lockless primitives, so locks need
     to be fast under contention, not just when uncontended.

(3) Not fitting our general design choice of aligning synchronisation
     primitives with POSIX -- a particularly important design choice in
     pthreads is not conflating mutual exclusion (locks) with process
     synchronisation (condition variables).  This semantic distinction improves
     our ability to use static and dynamic analysis (e.g., WITNESS, static lock
     order checking), but also affects our ability to offer priority
     propagation.  We could try and fix this in semaphores by providing more
     intent information (SEM_THIS_IS_A_MUTEX, SEM_THIS_IS_A_CONDVAR) but at
     that point I think we're much better off using the existing mutex, rwlock,
     and condition variable primitives directly, which gives us the information
     via C types.

The main reason sema(9) didn't go away was its use in our old ATA code; this 
is something that can and should be fixed, and the new ATA code does not take 
that perspective.  While I have no plans to do so myself, I'd be quite 
supportive of garbage collecting sema(9) in favour of requiring minor 
modifications to driver ports in order to better distinguish mutual exclusion 
and synchronisation, allowing WITNESS and other tools to properly analyse the 
behavior of imported code, and also to support future use of static analysis 
(e.g., the maturing Clang static lock analyser).

The only recent code I'm aware of to start using sema(9) is the Infiniband 
work you did; if you feel that it's not a significant burden to adapt it to 
use the native synchronisation primitives, then the main structural obstacle 
to moving away from sema(9) is eliminated.

Robert