I/OAT ... Coming Soon ?

[Andre Oppermann]

Jack Vogel wrote:
> On Nov 15, 2007 4:04 PM, Andre Oppermann <andre at freebsd.org> wrote:
>> Jack Vogel wrote:
>>> On Nov 14, 2007 5:01 PM, Wilkinson, Alex
>>> <Alex.Wilkinson at dsto.defence.gov.au> wrote:
>>>> Hi all,
>>>>
>>>> Curious, is I/OAT [http://www.intel.com/go/ioat/] coming to FreeBSD soon
>>>> ?
>>> LOL, I did a driver for the first version of I/OAT more than a year
>>> ago, submitted it and interest was half hearted.
>> IMHO the biggest drawback (design failure) is the polling requirement
>> for the work queue.  Ideally it would be structured like a NIC with
>> its DMA engine and rings.
> 
> I know, at the time I did this Chris Leech the Linux developer was having
> trouble with an interrupt design, so I figured I'd not bang my head against
> the wall unnecessarily and follow what he was doing :)
> 
> Since then however, work has been done and particularly with the CB2
> support I believe Linux now does use interrupts, so if I go back and rework
> the driver I will try and follow that path.

Actually it shouldn't be too hard.  I can provide you with the entry
points and a taskqueue entry.  The I/OAT support can be done just like
a driver serving the taskqueue.  The entry points for m_uiotombuf et al
would consider offloading if a certain size threshold is met and the
copy request is tagged with M_WAITOK.  The virtual to physical memory
lookup is probably costly though.  I don't think it supports the TLB
as that is only available within the CPU.  How is cache coherency
solved?  It'd be helpful if the documentation for I/OAT were available.

The Core2 and AMD64 CPUs have very high memory bandwidth with low
latency.  Is I/OAT still a significant enough win compared to the
implementation effort required?

-- 
Andre