igb dual-port adapter 1200Mbps limit - what to tune?

[Jack Vogel]

And why don't you use the driver in 7.4 PRERELEASE (2.0.7) and eliminate any
inevitable questions about fixes you don't have.

Jack


On Wed, Dec 1, 2010 at 10:06 AM, Vlad Galu <dudu at dudu.ro> wrote:

> On Wed, Dec 1, 2010 at 6:54 PM, Eugene Perevyazko <john at dnepro.net> wrote:
> > On Wed, Nov 10, 2010 at 01:04:28PM +0200, Eugene Perevyazko wrote:
> >>
> >> I have a router running RELENG_7 with two dual-port igbs - igb0 and igb1
> are on
> >> 82575 on intel s5520ur mb and igb2 and igb3 are on 82576 on ET dual-port
> card.
> >> 82576 is in 8x slot.
> >> Main traffic flows from igb0+igb1 to igb2+igb3, less traffic goes back.
> >> There's no traffic flow in directions igb0 - igb1 and igb2 - igb3.
> >>
> >> There are vlans on all interfaces.
> >>
> >> igb0 and igb1 are outbound links.
> >> igb2 and igb3 are connected to switch.
> >> CPU is E5620 at 2.4GHz, 8 cores, irqs bound to different cores skipping HT
> ones.
> >> Tried 2 queues and 1 queue per iface, neither hitting cpu limit.
> >>
> >> The problem is that traffic through igb2+igb3 is limited at around
> 1200Mbps Tx
> >> while I was hoping for 1600-1800Mbps Tx.
> >>
> >
> > I'd like to say that now this host is forwarding 1710 Mb/s @ 189 kpps in
> one direction + 411 Mb/s @ 140 kpps in reverse direction (30 minutes
> average).
> > The following changes were made:
> >  - no lagg in use
> >  - only one vlan left instead of 5 vlans
> >  - added motherboard module 'HP NC360T PCIe DP Gigabit Server Adapter
> (n1e5132)' with 2 em interfaces (only one used)
> >  - 3 interfaces are connected to hosts with patchcords, not through
> switch
> >  - hyperthreading turned off in bios
> >  - igb driver patched to reduce irq rate and make it tunable (patch
> follows)
> >  - hw.igb.num_queues=4, dev.igb.*.enable_aim=0,
> dev.igb.*.default_intrrate=4000 the last sysctl is added in patch)
> >  - igb queues manually repinned to different cores
> >  - ipfw rules minimized
>
> Can you please run a test with IPFW disabled? It's best to avoid using
> a firewall where throughput is the primary goal.
>
> >
> > igb driver used is 1.9.6/RELENG_7, igb.c version 1.3.2.12
> >
> > Can't say for sure whether there is a key change or is it all changes
> summed up,
> > sorry.
> >
> > --
> > Eugene Perevyazko
> >
> > _______________________________________________
> > freebsd-net at freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-net
> > To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
> >
>
>
>
> --
> Good, fast & cheap. Pick any two.
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>