Architecture vs. bus vs. device DMA cache coherency

[Warner Losh]

On Sep 10, 2013, at 9:43 AM, John-Mark Gurney wrote:

> Svatopluk Kraus wrote this message on Tue, Sep 10, 2013 at 13:19 +0200:
>> Even in DMA cache coherent architectures there could be not-coherent DMA
>> busses and/or devices. Thus, each bus and/or device should be described by
>> its bus_dma_tag and the tag should carry information about DMA cache
>> coherency.
> 
> I've thought about this a lot myself, and I'm not familar w/ a bus (that
> isn't main memory) or device that isn't cache coherent...  Most busses
> write to memory through an arbiter (north bridge or cpu/soc) that does
> the proper read/modify/write cycles to get the memory there.  I have
> not heard of another bus/device that does their own read/modify/write
> cycles to get their writes to memory.  Can you name a current bus/device
> that does this?
> 
> Our busdma system does have issues that if you try to dma to say,
> video memory, we don't handle that (well) because we assume that all
> memory is a flat space and belongs to nexus, but this isn't always
> correct.
> 
> If you have an architecture like this, can you please tell use which
> system you are trying to fix?

I know that MIPS has different kinds of coherency as well that we generally map to coherent (meaning coherent enough for the users) or incoherent. Something to keep in mind.

I generally like the idea, but don't have enough experience with kind of hardware to know if the design is a good one.

Warner