igb(4) Pondering a bind to cpu patch

[John Baldwin]

On Tuesday, April 24, 2012 8:11:07 pm Sean Bruno wrote:
> http://people.freebsd.org/~sbruno/if_igb.c.txt
> 
> Scenario I've just seen:
> 
> 8 core machine
> 2 igb(4) interfaces
> set num_queues=4
> 
> igb0:0 --> cpu0
> igb0:1 --> cpu1
> igb0:2 --> cpu2
> igb0:3 --> cpu3
> 
> igb1:0 --> cpu0
> igb1:1 --> cpu1
> igb1:2 --> cpu2
> igb1:3 --> cpu3
> 
> I suspect, that we need a static global to keep track of what cpu last
> was last bound to a queue.  My patch does do this, but I don't know if
> its the right thing.
> 
> Since I'm doing multiple interfaces, I need to make sure I don't
> schedule a queue to a non existent cpu, so take a modulo of the counter
> and the number of cpus in the box.  
> 
> Perhaps not the most elegant solution, but its a thing?

CPU IDs are not guaranteed to be dense.  However, you can use CPU_FIRST() and 
CPU_NEXT() with your static global instead.

OTOH, if igb were to just leave the interrupts alone instead of binding them 
by hand, they would get round-robin assigned among available cores already.  I 
think in this case the best approach might be to add a tunable to disable 
igb's manual binding and instead let the default system round-robin be 
preserved.

-- 
John Baldwin