FreeBSD 10G forwarding performance @Intel

[Alexander V. Chernikov]

On 03.07.2012 20:55, Luigi Rizzo wrote:
> On Tue, Jul 03, 2012 at 08:11:14PM +0400, Alexander V. Chernikov wrote:
>> Hello list!
>>
>> I'm quite stuck with bad forwarding performance on many FreeBSD boxes
>> doing firewalling.
> ...
>> In most cases system can forward no more than 700 (or 1400) kpps which
>> is quite a bad number (Linux does, say, 5MPPs on nearly the same hardware).
>
> among the many interesting tests you have run, i am curious
> if you have tried to remove the update of the counters on route
> entries. They might be another severe contention point.

21:47 [0] m at test15 netstat -I ix0 -w 1
             input          (ix0)           output
    packets  errs idrops      bytes    packets  errs      bytes colls
    1785514 52785     0  121318340    1784650     0  117874854     0
    1773126 52437     0  120701470    1772977     0  117584736     0
    1781948 52154     0  121060126    1778271     0   75029554     0
    1786169 52982     0  121451160    1787312     0  160967392     0
21:47 [0] test15# sysctl net.rt_count=0
net.rt_count: 1 -> 0
    1814465 22546     0  121302076    1814291     0   76860092     0
    1817769 14272     0  120984922    1816254     0  163643534     0
    1815311 13113     0  120831970    1815340     0  120159118     0
    1814059 13698     0  120799132    1813738     0  120172092     0
    1818030 13513     0  120960140    1814578     0  120332662     0
    1814169 14351     0  120836182    1814003     0  120164310     0

Thanks, another good point. I forgot to merge this option from andre's 
patch.

Another 30-40-50kpps to win.


+u_int rt_count  = 1;
+SYSCTL_INT(_net, OID_AUTO, rt_count, CTLFLAG_RW, &rt_count, 1, "");

@@ -601,17 +625,20 @@ passout:
         if (error != 0)
                 IPSTAT_INC(ips_odropped);
         else {
-               ro.ro_rt->rt_rmx.rmx_pksent++;
+               if (rt_count)
+                       ro.ro_rt->rt_rmx.rmx_pksent++;
                 IPSTAT_INC(ips_forward);
                 IPSTAT_INC(ips_fastforward);


>
> cheers
> luigi
>


-- 
WBR, Alexander