busdma buffer management enhancements - call for review and test

[Mark Tinguely]

On Tue, Sep 4, 2012 at 11:58 AM, Ian Lepore
<freebsd at damnhippie.dyndns.org> wrote:
> The attached set of patches enhances the ARM v4/v5 busdma management of
> small DMA buffers, especially for BUS_DMA_COHERENT and uncacheable
> memory.  The existing implementation uses malloc(9) to allocate buffers,
> then uses arm_remap_nocache() to make the page(s) containing the buffers
> uncacheable, even when those pages contain data other than DMA buffers.
>
> The attached patches address this by:
>
>       * Adding support for pmap_page_set_memattr() and
>         VM_MEMATTR_UNCACHEABLE to the ARM v4 pmap implemenation.


>       * Adding new common code usable by any platform that uses uma(9)
>         to manage pools of power-of-two sized buffers, keeping them
>         sized and aligned as required to help maintain cache coherency.
>       * Modifying the busdma implementation to use the new pool manager
>         to manage pools of both regular and uncacheable buffers, and
>         also to use uma(9) to manage a pool of map structures.
>       * Using knowledge of the alignment and padding of pool-allocated
>         buffers to avoid doing partial cache line flushes on those
>         buffers.
>
> I'm CC'ing freebsd-mips because I think these patches can be adapted to
> the MIPS busdma implementation as well.  The new busdma_bufalloc.[ch]
> code isn't arm-specific, and shouldn't live in sys/arm/arm/, but I don't
> know where it should go.
>
> Once we have ARM and MIPS able to efficiently manage small cache-aligned
> DMA buffers, the stage is set to begin updating device drivers to
> allocate buffers individually, rather than allocating huge chunks and
> sub-dividing them, possibly violating cache line boundaries in doing so.
>
> I've been running these for a couple days on DreamPlug and Atmel
> hardware (the latter in an 8.x environment) without problems.
>
> -- Ian


How about having a per processor cache line definition rather than using:

+#define        MIN_ZONE_BUFSIZE        32

For those archs that have a 64 byte cache line.

I like the separation of the regular and BUS_DMA_COHERENT or
BUS_DMA_NOCACHE. I never liked turning the cache off the entire page
for a few DMA buffers.

--Mark.