irq cpu binding

[Adrian Chadd]

On 28 March 2015 at 16:05, Slawa Olhovchenkov <slw at zxy.spb.ru> wrote:
> On Sat, Mar 28, 2015 at 03:49:48PM -0700, Adrian Chadd wrote:
>
>> You should totally join #bsdcode on efnet and ask me about it. :)
>
> I am totaly don't use IRC (last 20 years).
> May be skype?

Heh, IRC is better. There are more FreeBSD people in the channel. :)

>> on RSS, this is what would happen:
>>
>> * ALL NICs RSS BUCKET 0 -> core 0
>> * ...
>> * ALL NICs RSS BUCKET 7 -> core 7
>
> My expirens: this is worse vs dedicated core (one core handeled only
> one bucket of one NIC).

The only reason(s) this becomes problematic is if things preempt other
things on that CPU.
Hopefully enough work gets done in each interrupt run - but, maybe the
scheduler is doing something odd and interleaving all the
supposedly-equivalent-ithreads based on what's blocking in locks and
what isn't. It's worth digging into.

Not only that, but I also do handle the case of fragments going to the
"wrong" queue - then getting reassembled and reinjected back into the
right RSS CPU. That way things are correctly in-order.

>
>> Now, that's not really 100% optimal for NUMA and multiple PCIe
>> controllers, but we're not there yet.
>>
>> Hopefully I can twist/cajole navdeep @ chelsio to continue doing a
>> little more RSS work so I can teach cxgbe/cxl about RSS configuration,
>> but ixgbe, igb and ixl all do the above when RSS is enabled.
>
> Most part of my setup use cxgbe.

Ok.

Well, that (and other stuff) will happen at the speed of "adrian's
doing this for fun as his home project", so if you/others would like
to help out then please do. I'd like to get this stuff very much done
and in -11 before it's released next year.



-adrian