Aggregating many ports into one for tcpdump server. (also sampling before libpcap)

[Robert Watson]

On Sat, 8 Dec 2007, Peter Wood wrote:

> I'd prefer to use sampling rather then just accepting kernel droped packets 
> to ensure fair selection over a time period, rather then only collecting the 
> start of that period and then nothing else.
>
> I'd be willing to look into implementing that perhaps in the same way that 
> Juniper Networks do for their sampling, ie. a maximum number of packets to 
> be sampled in a second, how often to sample in terms of packets and then 
> when sampling how many packets it should sample.

Something that would be quite easy to do is add a "select 1 in (n) matched 
packets for sampling", and just adding a counter to each BPF device that goes 
up with each seen (but not necessarily sampled) packet.  You could imagine 
more complex solutions that were more capable.  Regardless, I suspect that the 
mechanism to implement a sampling scheme for BPF would be pretty straight 
forward and I'd be happy to lend a hand.

If you take a look at points in the bpf.c that call bpf_filter() and 
catchpacket(), that should get you the points of interest for such a decision. 
For simple rate control, I suppose one wants to do the sampling before calling 
the BPF filter to avoid burning CPU processing the filter for packets you 
won't accept anyway, in as much as that doesn't modify the semantics of the 
sampling and filtering models.

Robert N M Watson
Computer Laboratory
University of Cambridge