Intel 82580 lagg(4) problem

[Adrian Chadd]

Hi,

There's a lagg/lacp option to not hash based on the incoming assigned
flowid on the incoming mbuf when transmitting but instead calculate its own
hash for outbound transmission. It's net.link.lagg.0.use_flowid . Set it to
0 and retry.

Thanks,


-adrian



On 30 September 2013 02:10, Andriy Kopystyansky <anri at polynet.lviv.ua>wrote:

> Greetings!
> We've got a strange problem with lagg(4) interfaces built on Intel's 82580
> chipset igb (dual port 1Gb card).
> We cannot send over lagg interface more than ~1Gb/s, yet receiving ~2GB
> over it.
> Looks like something indicating the problem can be seen here:
>
> anri at host:[8:13]~#ifstat -i lagg0 -i igb1 -i igb3 1
>       lagg0                igb1                igb3
>  KB/s in  KB/s out   KB/s in  KB/s out   KB/s in  KB/s out
>  9116.50  26515.16   4147.70  28871.43   5004.86  23683.31
>  8423.08  26544.62   3853.22  28980.60   4594.21  23520.97
>  8796.48  26395.28   4248.46  28344.00   4567.42  23978.17
>
> Note (IN) traffic - everything's ok, about 50% on each igb interfaces and
> total summary on lagg0.
> But (OUT) traffic looks weird - there is more traffic on the single igb1
> than on lagg0!
> Tried with default driver came with the system, also the new one
> igb-2.3.10 from Intel site - no luck.
>
> Testing lagg interface built on other cards (em, bce) on the same machine
> shows expected normal behavior for both directions.
> ------------------------------**---------
> anri at host:[9:02]~#uname -a
> FreeBSD host 9.1-RELEASE-p4 FreeBSD 9.1-RELEASE-p4 #5: Fri Sep 20 14:52:18
> EEST 2013     anri at host:/usr/obj/usr/src/**sys/NODEONE  amd64
> ------------------------------**--------
> Interfaces:
> lagg0: flags=8843<UP,BROADCAST,**RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
> 1500
>         options=401bb<RXCSUM,TXCSUM,**VLAN_MTU,VLAN_HWTAGGING,JUMBO_**
> MTU,VLAN_HWCSUM,TSO4,VLAN_**HWTSO>
>         ether 90:e2:ba:36:95:1d
>         inet 172.16.0.1 netmask 0xffffffc0 broadcast 172.16.0.63
>         media: Ethernet autoselect
>         status: active
>         laggproto lacp lagghash l2,l3,l4
>         laggport: igb3 flags=1c<ACTIVE,COLLECTING,**DISTRIBUTING>
>         laggport: igb1 flags=1c<ACTIVE,COLLECTING,**DISTRIBUTING>
>
> igb1: flags=8843<UP,BROADCAST,**RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
> 1500
>         options=401bb<RXCSUM,TXCSUM,**VLAN_MTU,VLAN_HWTAGGING,JUMBO_**
> MTU,VLAN_HWCSUM,TSO4,VLAN_**HWTSO>
>         ether 90:e2:ba:36:95:1d
>         media: Ethernet autoselect (1000baseT <full-duplex>)
>         status: active
>
> igb3: flags=8843<UP,BROADCAST,**RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
> 1500
>         options=401bb<RXCSUM,TXCSUM,**VLAN_MTU,VLAN_HWTAGGING,JUMBO_**
> MTU,VLAN_HWCSUM,TSO4,VLAN_**HWTSO>
>         ether 90:e2:ba:36:95:1d
>         media: Ethernet autoselect (1000baseT <full-duplex>)
>         status: active
>
> ------------------------------**---------
> Hardware: IBM 3550 with 2x Intel(R) Xeon(R) CPU  E5430  @ 2.66GHz (amd64)
> pciconf -lv:
> igb1 at pci0:19:0:1:       class=0x020000 card=0x12b28086 chip=0x15168086
> rev=0x01 hdr=0x00
>     vendor     = 'Intel Corporation'
>     device     = '82580 Gigabit Network Connection'
>     class      = network
>     subclass   = ethernet
>
> ------------------------------**---------
> /boot/loader.conf:
> hw.igb.rxd=4096
> hw.igb.txd=4096
> hw.igb.max_interrupt_rate=**32000
> hw.igb.num_queues=2
> net.isr.maxthreads=8
> net.isr.dispatch=direct
> net.isr.bindthreads=1
> net.link.ifqmaxlen=10240
>
> /etc/sysctl.conf: (relevant part)
>
> dev.igb.1.rx_processing_limit=**-1
> dev.igb.3.rx_processing_limit=**-1
> net.inet.ip.intr_queue_maxlen=**512
> net.route.netisr_maxqlen=4096
> net.inet.ip.redirect=0
> net.inet.ip.fastforwarding=1
> kern.maxfiles=204800
> kern.maxfilesperproc=200000
> kern.ipc.nmbclusters=204800
> kern.ipc.maxsockets=204800
>
> Any tips, advices?
>
> thanks in advance, with best regards,
> Andriy Kopystyansky
>
>
> ______________________________**_________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/**mailman/listinfo/freebsd-net<http://lists.freebsd.org/mailman/listinfo/freebsd-net>
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@**freebsd.org<freebsd-net-unsubscribe at freebsd.org>
> "
>