Question about wmb() and rmb() and task switching

[Konstantin Belousov]

On Fri, Sep 23, 2016 at 01:46:15PM +0200, Hans Petter Selasky wrote:
> Hi,
> 
> Does use of wmb() and rmb() for amd64 as defined in 
> sys/amd64/include/atomic.h required use of critical_enter()/critical_exit().
> 
> I was looking at the code in sys/amd64/amd64/cpu_switch.S which switches 
> between threads and I don't see any "sfence" instructions in there.
> 
> Given the following piece of dummy code:
> 
> var_a = 1;
> var_b = 2;
> wmb();
> 
> If there is a task switch between writing var_a and var_b so that the 
> thread in question continues executing on another core, can it happen 
> that the write to var_a is not flushed when wmb() is executed?
> 
> var_a = 1;
> <task switch to different CPU core>
> var_b = 2;
> wmb();

cpu_switch() guarantees that the context switch behaves as a full
barrier. In other words, all side-effects which are before context
switch in the program order, become globally visible after it (if
address spaces are switched), context switch behaves as the strongest
barrier among all provided by the hardware.

For amd64, there are more than one serialization instructions executed
during the switch, including the access to cr3 and thread unlock for old
thread.  If the context switch is involuntary, then the interrupt itself
also provides serialization point.

Note that FreeBSD uses C11 memory model of load acquire/store release,
not the linux-like rmb/wmb. atomic acq/rel ops are strongly preferred
over the *mb(), which mostly exists as the initial driver porting aid.