Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp]

[Mike Tancsa]

At 04:04 AM 6/29/2008, Paul wrote:
>This is just a question but who can get more than 400k pps 
>forwarding performance ?


OK, I setup 2 boxes on either end of a RELENG_7 box from about May 
7th just now, to see with 2 boxes blasting across it how it would 
work.  *However*, this is with no firewall loaded and, I must enable 
ip fast forwarding. Without that enabled, the box just falls over.

even at 20Kpps, I start seeing all sorts of messages spewing to route 
-n monitor


got message of size 96 on Mon Jun 30 15:39:10 2008
RTM_MISS: Lookup failed on this address: len 96, pid: 0, seq 0, errno 
0, flags:<DONE>
locks:  inits:
sockaddrs: <DST>
  default

I am starting to wonder if those messages are the results of 
corrupted packets the machine just cant keep up with ?


CPU is

CPU: Intel(R) Xeon(R) CPU            3070  @ 2.66GHz (2660.01-MHz 
686-class CPU)


            input        (Total)           output
    packets  errs      bytes    packets  errs      bytes colls
     611945     0   77892098     611955     0   77013002     0
     616727     0   78215508     616742     0   77303454     0
     617066     0   78162130     617082     0   77238434     0
     618238     0   78302314     618225     0   77377582     0
     617035     0   78141000     617038     0   77215672     0
     617625     0   78225600     617588     0   77301734     0
     616190     0   78017320     616165     0   77091774     0
     615583     0   78064130     615628     0   77152800     0
     617662     0   78254388     617658     0   77332340     0
     618000     0   78269912     617950     0   77344554     0
     617248     0   78183136     617315     0   77259588     0
     617325     0   78204566     617289     0   77282094     0
     618391     0   78337734     618357     0   77413756     0
     616025     0   78116070     616082     0   77203116     0


To generate the packets, I am just using 
/usr/src/tools/tools/netblast  on 2 endpoints starting at about the same time

# ./netblast 10.10.1.2 500 100 40

start:             1214854131.083679919
finish:            1214854171.084668592
send calls:        20139141
send errors:       0
approx send rate:  503478
approx error rate: 0


# ./netblast 10.10.1.3 500 10 40

start:             1214854273.882202815
finish:            1214854313.882319031
send calls:        23354971
send errors:       18757223
approx send rate:  114943
approx error rate: 0

The box in the middle doing the forwarding

1[spare-r7]# ifconfig -u
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
         options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
         ether 00:1b:21:08:32:a8
         inet 10.20.1.1 netmask 0xffffff00 broadcast 10.20.1.255
         media: Ethernet autoselect (1000baseTX <full-duplex>)
         status: active
em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
         options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
         ether 00:1b:21:08:32:a9
         inet 192.168.43.193 netmask 0xffffff00 broadcast 192.168.43.255
         media: Ethernet autoselect (100baseTX <full-duplex>)
         status: active
em3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
         options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
         ether 00:30:48:90:4c:ff
         inet 10.10.1.1 netmask 0xffffff00 broadcast 10.10.1.255
         media: Ethernet autoselect (1000baseTX <full-duplex>)
         status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
         inet 127.0.0.1 netmask 0xff000000


I am going to try a few more tests with and without, firewall rules 
etc as well as an updated kernel to RELENG_7 as of today and see how that goes.

         ---Mike