Kernelspace C11 atomics for MIPS

[Warner Losh]

On Jun 3, 2013, at 10:15 PM, Patrick Kelsey wrote:

> On Tue, Jun 4, 2013 at 12:08 AM, Patrick Kelsey <kelsey at ieee.org> wrote:
>> On Mon, Jun 3, 2013 at 11:57 PM, Adrian Chadd <adrian at freebsd.org> wrote:
>>> On 3 June 2013 20:55, Juli Mallett <jmallett at freebsd.org> wrote:
>>> 
>>>> To drain the pipeline on certain deficient (and mostly older) CPUs by way of
>>>> guesswork and a little vague magic.  Most CPUs we support, I would guess, do
>>>> not need this, and it continues to exist solely for hysterical reasons.
>>> 
>>> How can I turn it off for my compiles?
>>> 
>>>> I've certainly gotten rid of them and some other cargo cult synchronization
>>>> on Octeon for testing and had it survive under considerable load, and
>>>> occasionally with some slight speedups (for some more commonly-used or
>>>> slower things than Just a Bunch Of NOPs.)
>>> 
>>> Right. Well, since it's happening on every inlined lock, it's a bit silly.
>>> 
>>>> The trouble is that proving they aren't necessary requires being rigorous
>>>> and careful in understanding documentation and errata, and FUD about their
>>>> possible necessity is somewhat-intimidating.  It's not an easy kind of
>>>> corruption/unreliability/etc., to prove the lack of empirically.
>>> 
>>> I've checked the diassembly from gcc-4.mumble on linux; it doesn't
>>> include NOPs like this as far as I can tell.
>>> 
>> 
>> The sync + 8 nops is coming from the definition of mips_sync() in
>> sys/mips/include/atomic.h.

They came from the old mips2 branch, which may have been from the Juniper code merge, or maybe not.

>> I agree with Juli that it appears to be a manual pipeline-flush
>> holdover from earlier days - I'm guessing there's 8 nops because the
>> R4000/4400 had both the sync instruction and an 8-stage pipeline.  I'm
>> further guessing this was an attempt at providing stronger ordering
>> semantics than the sync instruction itself for the following
>> mb()/wmb()/rmb() definitions that use it, as the sync instruction
>> definition doesn't restrict execution of the before/after loads/stores
>> with respect to the sync instruction itself.
> 
> Forgot to emphasize that this particular bit of old-school
> nop-counting is either pointless or a latent hazard - 8 does not cover
> the deepest MIPS pipeline around, then there's superscalar issue to
> consider - so I think it's either unnecessary or insufficient.  So
> far, that's all criticism and no solution :/

Yes, there's new nops for these situations starting in mips32r2 and mips64r2 ISAs.

I think that this originated in the mips-jnpr merge, but can't find the old branches anywhere...

Warner