Request for feedback on common data backstore in the kernel

[Hans Petter Selasky]

Hi Scott,

The discussion has been moved to "freebsd-arch at freebsd.org". Please only reply 
there next time.

On Wednesday 26 September 2007, Scott Long wrote:
> Hans Petter Selasky wrote:
> > Hi,
> >
> > Please keep me CC'ed, hence I'm not on all these lists.
> >
> > In the kernel we currently have two different data backstores:
> >
> > struct mbuf
> >
> > and
> >
> > struct buf
> >
> > These two backstores serve two different device types. "mbufs" are for
> > network devices and "buf" is for disk devices.
> >
> > Problem:
> >
> > The current backstores are loaded into DMA by using the BUS-DMA
> > framework. This appears not to be too fast according to Kip Macy. See:
> >
> > http://perforce.freebsd.org/chv.cgi?CH=126455
>
> Busdma isn't fast enough for 1Gb and 10Gb network drivers that are
> trying to max out their packet rates.  It's still fine for storage
> drivers and other slow/medium speed device drivers, like USB and
> Firewire.  I am working on techniques to make the API easier to use
> and the implementation fast enough for the aforementioned devices.

That's great!

>
> > Some ideas I have:
> >
> > When a buffer is out out of range for a hardware device and a data-copy
> > is needed I want to simply copy that data in smaller parts to/from a
> > pre-allocated bounce buffer. I want to avoid allocating this buffer when
> > "bus_dmamap_load()" is called.
>
> I think you've missed the point of having architecture portable drivers.
> John-Mark describes this further. 

I use the bus_dma framework to allocate and sync all DMA memory, and I assume 
that is correct.

> It also makes little sense to push 
> the responsibility for handling bounce buffers out of a central
> subsystem and back into every driver.

I'm thinking about putting some wrappers around the "bus_dmamap_load()" 
function like:

void usbd_rx_buf_load(struct usbd_xfer *xfer, void *buf,
  uint32_t framebuffer_offset, uint32_t framebuffer_index, uint32_t len);

void usbd_tx_buf_load(struct usbd_xfer *xfer, void *buf,
  uint32_t framebuffer_offset, uint32_t framebuffer_index, uint32_t len);

But I haven't figured out all the details yet. The "usbd_xxx_load()" functions 
should automagically figure out what is best to do and it won't be more than 
a few lines of code.

--HPS