atomic ops

[Konstantin Belousov]

On Tue, Oct 28, 2014 at 03:52:22AM +0100, Mateusz Guzik wrote:
> As was mentioned sometime ago, our situation related to atomic ops is
> not ideal.
> 
> atomic_load_acq_* and atomic_store_rel_* (at least on amd64) provide
> full memory barriers, which is stronger than needed.
x86 atomic_store_rel() does not establish any cpu barrier, due to the
already provided guarantees of the architecture.

> 
> Moreover, load is implemented as lock cmpchg on var address, so it is
> addditionally slower especially when cpus compete.
> 
> On amd64 it is sufficient to place a compiler barrier in such cases.
> 
> Next, we lack some atomic ops in the first place.
> 
> Let's define some useful terms:
> smp_wmb - no writes can be reordered past this point
> smp_rmb - no reads can be reordered past this point
> 
> With this in mind, we lack ops which would guarantee only the following:
> 
> 1. var = tmp; smp_wmb();
> 2. tmp = var; smp_rmb();
> 3. smp_rmb(); tmp = var;
> 
> This matters since what we can use already to emulate this is way
> heavier than needed on aforementioned amd64 and most likely other archs.
> 
> It is unclear to me whether it makes sense to alter what
> atomic_load_acq_* are currently doing.
I still think that our load/stores, comparing with the classic definition
of the operations, are ordered, i.e. what is called sequential consistent
in the C standard.  I have no idea if we want this property, or is it
used really.  The kern_intr.c (ab)uses load in this way.

> 
> The simplest thing would be to just introduce aforementioned macros.
> 
> Unfortunately I don't have any ideas for new function names.
> 
> I was considering stealing consumer/producer wording instead of acq/rel,
> but that does not help with case 1.
> 
> Also there is no common header for atomic ops.
> 
> I propose adding sys/atomic.h which includes machine/atomic.h. Then it
> would provide atomic ops missing from md header implemented using what
> is already there.
> 
> For an example where it could be useful see
> https://svnweb.freebsd.org/base/head/sys/sys/seq.h?view=markup
> 
> Comments?
> 
> And yes, I know that:
> - atomic_load_acq_rmb_int is a terrible name and I'm trying to get rid
>   of it
> - seq_consistent misses a read memory barrier, but in worst case this
>   will result in spurious ENOTCAPABLE returned. security problem of
>   circumventing capabilities is plugged since seq is properly re-checked
>   before we return
> 
> -- 
> Mateusz Guzik <mjguzik gmail.com>