Architecture vs. bus vs. device DMA cache coherency

[Svatopluk Kraus]

On Tue, Sep 10, 2013 at 5:43 PM, John-Mark Gurney <jmg at funkthat.com> wrote:

> Svatopluk Kraus wrote this message on Tue, Sep 10, 2013 at 13:19 +0200:
> > Even in DMA cache coherent architectures there could be not-coherent DMA
> > busses and/or devices. Thus, each bus and/or device should be described
> by
> > its bus_dma_tag and the tag should carry information about DMA cache
> > coherency.
>
> I've thought about this a lot myself, and I'm not familar w/ a bus (that
> isn't main memory) or device that isn't cache coherent...  Most busses
> write to memory through an arbiter (north bridge or cpu/soc) that does
> the proper read/modify/write cycles to get the memory there.  I have
> not heard of another bus/device that does their own read/modify/write
> cycles to get their writes to memory.  Can you name a current bus/device
> that does this?
>
> Our busdma system does have issues that if you try to dma to say,
> video memory, we don't handle that (well) because we assume that all
> memory is a flat space and belongs to nexus, but this isn't always
> correct.
>
> If you have an architecture like this, can you please tell use which
> system you are trying to fix?


Well, bus DMA implementation is MD code. We have taken into account ARM
architecture mainly and tried to cover not-coherent ARMs, ARMs with ACP,
and coherent ARMs. However, bus_dma_tag_t is common interface, so we've
spread our thoughts and added to our model even hypotetical cases.

You are right that devices with own memory on them, which is not accessed
thru a bus from device side, are an issue. Like fast data (video) grabbers
with dual-ported RAMs.

Svatopluk Kraus