Zero-copy BPF update (was: Re: Aggregating many ports into one for tcpdump server.)

[Robert Watson]

On Wed, 5 Dec 2007, Vlad GALU wrote:

>>> I've had several reports of significantly improved packet capture rates at 
>>> high speeds with it, but it's not yet in the tree because we feel it needs 
>>> more evaluation and review.  I hope to ship some form of zero-copy BPF 
>>> buffer support in FreeBSD 8, and possibly even MFC it.  Any feedback you 
>>> might have would be most helpful.
>>
>> Having sent you the patch, I should have let you know that you'll need to:
>>
>> - Add options BPF_ZEROCOPY to your kernel configuration to enable the
>>   zero-copy buffering mode.
>>
>> - Make sure the kernel and libpcap are rebuild following the application of
>>   the patch and dropping in the tarball.
>>
>> - setenv BPF_ZERO_COPY before running tcpdump or other BPF-based tools to
>>   enable the zero-copy buffer mode.
>>
>> The patch includes both kernel changes (abstract the buffer model, add a 
>> new buffer model) and user space changes (updated libpcap to speak the new 
>> model, selected right now with the environmental variable).  Presumably if 
>> merged, zero-copy BPF buffers would be used by default via libpcap if 
>> present in the kernel, but right now this is all for evaluation purposes.
>
> Thanks, Robert! I'll start running a few tests next week, I'm waiting for 
> some hardware to arrive first.

I've put up an updated tarball based on some recent changes here:

   http://www.watson.org/~robert/freebsd/20071226-zcopybpf.tgz

The main changes since this last drop are:

- BPF_ZERO_COPY environmental variable renamed to BPF_ZEROCOPY to match kernel
   option name.

- libpcap support for zero-copy BPF buffers reworked to avoid unconditional
   call to select() for each buffer when there's already a pending buffer
   available to use; in general, avoid system calls entirely when there's data
   already waiting, only use system calls when there isn't a completed buffer
   to work on next.

- Comments cleanup and some code cleanup.

- A README to provide a little more guidance on getting it working. :-)

You will need to "make clean ; make ; make install" in the modified libpcap 
against, as the size of pcap_t has changed.  In principle "make ; make 
install" should DTRT, but it appears not to for me.

Robert N M Watson
Computer Laboratory
University of Cambridge