igb dual-port adapter 1200Mbps limit - what to tune?

[Aleksandr A Babaylov]

On Thu, Nov 11, 2010 at 08:05:40PM +0200, Eugene Perevyazko wrote:
> On Thu, Nov 11, 2010 at 12:49:52PM +0200, Eugene Perevyazko wrote:
> > On Thu, Nov 11, 2010 at 01:47:02AM +0100, Ivan Voras wrote:
> > > On 11/10/10 12:04, Eugene Perevyazko wrote:
> > > 
> > > >Tried 2 queues and 1 queue per iface, neither hitting cpu limit.
> > > 
> > > Are you sure you are not hitting the CPU limit on individual cores? Have 
> > > you tried running "top -H -S"?
> > > 
> > Sure, even with 1queue per iface load is 40-60% on busy core, with 2 queues it was much lower.
> > Now I've got the module for mb with 2 more ports, going to see if it helps.
> The IO module has em interfaces on it and somehow I've already got 2 panics
> after moving one of vlans to it.
> 
> In the mean time, can someone explain me what is processed by threads marked 
> like "irq256: igb0" and "igb0 que". May be understanding this will let me
> pin those threads to cores more optimally.
> There are (hw.igb.num_queues+1) "irq" threads and (hw.igb.num_queues)
> "que" threads. Now I just pin them sequentially to even cores (odd ones are HT).
As far as I understand, you are not right about HT cores.
Try switch HT off and do not use HT in routers
in usual cases.

> Now I use hw.igb.num_queues=2, and with traffic limited to 1200Mbits the busiest core is still 60% idle...
> 
> 
> 
> -- 
> Eugene Perevyazko
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"