New optimized soreceive_stream() for TCP sockets, proof of concept

[John Baldwin]

On Monday 05 March 2007 13:29, Robert Watson wrote:
> On Mon, 5 Mar 2007, Andrew Gallatin wrote:
> 
> > With the patch, we finally seem to be performance competative on the 
receive 
> > side with Linux x86_64 and Solaris/amd64 on this same hardware. Both of 
> > those OSes do much better (saturate the link with jumbos) when CPU 
affinity 
> > is used to bind the interrupt handler and netserver process to different 
> > cores on the same socket.  I imagine FreeBSD may be able to do even better 
> > if it ever grows CPU affinity support for both interrupt handlers and 
> > processes.  With the patch, it performs at least as well, if not better 
> > than, Solaris and Linux do without CPU affinity.
> 
> I don't have numbers in front of me, and am currently packing for a trip to 
> Tokyo so won't find them before traveling, but my experience has been that 
> binding the ithread to a specific CPU is very helpful in improving receive 
> performance.  You can slap a sched_bind(0) into the interrupt handler the 
> first time it runs and it should stick appropriately, and add a sysctl to 
> sched_bind() for a user process as a hack to test it out.
> 
> John has a patch that pins interrupt threads, etc, not sure what the status 
of 
> that is.  CC'd.

Tested and around for over a year.  Sent to people several times but no 
benchmarking has resulted.  It lives in p4 in //depot/user/jhb/intr/...

I've just regenerated the patch at 
http://www.FreeBSD.org/~jhb/patches/intr_bind.patch (same URL as the last N 
times I've posted it).  I do now have access again to the sources for a 
simple intr_bind userland tool that lets you move assign the CPU for a given 
interrupt.  There is currently no way to see which CPU an interrupt is 
assigned to outside of 'show intr' in ddb or scraping a verbose dmesg, but 
that can always be added later.  What is there now is probably sufficient for 
testing.

-- 
John Baldwin