i386/61858: bus_dmamap_sync with BUS_DMASYNC_POSTREAD needs memory barrier.

[Stephan Uphoff]

>Number:         61858
>Category:       i386
>Synopsis:       bus_dmamap_sync with BUS_DMASYNC_POSTREAD needs memory barrier.
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-i386
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sat Jan 24 17:40:09 PST 2004
>Closed-Date:
>Last-Modified:
>Originator:     Stephan Uphoff
>Release:        current
>Organization:
>Environment:
N/A
>Description:
bus_dmamap_sync is basically a no-op on i386 for PCI.
However since modern i386 CPUs can issue speculative out of order readaheads a memory barrier is required for BUS_DMASYNC_POSTREAD

Example: 

Device uses DMA for data (D) and control (C) memory.
CPU tests if DMA finished by checking (C)

Some ugly pseudo code:

bus_dmamap_sync(.....,BUS_DMASYNC_POSTREAD);  /* Sync (C) Control */
if (C)
	{
	bus_dmamap_sync(.....,BUS_DMASYNC_POSTREAD);  /* Sync (D) DATA */
	var = D;
	....

The current  bus_dmamap_sync(.....,BUS_DMASYNC_POSTREAD) does not act
as a memory barrier as required by the manual page.

This means the CPU can reorder the read accesses to the memory.
(No-op bus_dmamap_sync removed) 

	speculative_preload  = D;
                    <---- Race condition if DMA happens here
	if (C)
	    {
	     var = speculative_preload;


var can contain invalid data as it might have
been loaded before C became valid.
>How-To-Repeat:
      
>Fix:
Add a memory barrier.

Import recent changes to bus_dmamap_sync from NetBSD.

Linux has an interesting solution (link section trick) that dynamically
patches the executable to use the fastest memory barrier
supported by the current CPU.
>Release-Note:
>Audit-Trail:
>Unformatted: