Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp]

[Ingo Flaschberger]

Dear Paul,

>>> what could cause this?
>> 
>> *) kern.polling.idle_poll enabled?
>> *) kern.polling.user_frac ?
>> *) kern.polling.reg_frac ?
>> *) kern.polling.burst_max ?
>> *) kern.polling.each_burst ?
>
> I tried tons of different values for these and nothing made any significant 
> difference.
> Idle polling makes a difference, allows more pps, but still errors.
> Without idle polling it seems PPS is limited to HZ * descriptors, or 1000 * 
> 256 or 512
> but 1000 * 1024 is the same as 512..  4000 * 256 or 2000 * 512 works but 
> starts erroring 600kpps (SMP right now but it happens in UP too)

I have patched src/sys/kern/kern_poll.c to support higher burst_max 
values:
#define MAX_POLL_BURST_MAX      10000

When setting kern.polling.burst_max to higher values, the server reach a 
point, where cpu-usage goes up without load, so try to keep below this 
values. I also have set the network card to 4096 rx-"ram", to have more 
room for late polls.

Kind regards,
 	Ingo Flaschberger