New optimized soreceive_stream() for TCP sockets, proof of concept

[Andrew Gallatin]

Robert Watson writes:
 > On Mon, 5 Mar 2007, Andrew Gallatin wrote:
 > 
 > > With the patch, we finally seem to be performance competative on the receive 
 > > side with Linux x86_64 and Solaris/amd64 on this same hardware. Both of 
 > > those OSes do much better (saturate the link with jumbos) when CPU affinity 
 > > is used to bind the interrupt handler and netserver process to different 
 > > cores on the same socket.  I imagine FreeBSD may be able to do even better 
 > > if it ever grows CPU affinity support for both interrupt handlers and 
 > > processes.  With the patch, it performs at least as well, if not better 
 > > than, Solaris and Linux do without CPU affinity.
 > 
 > I don't have numbers in front of me, and am currently packing for a trip to 
 > Tokyo so won't find them before traveling, but my experience has been that 
 > binding the ithread to a specific CPU is very helpful in improving receive 
 > performance.  You can slap a sched_bind(0) into the interrupt handler the 
 > first time it runs and it should stick appropriately, and add a sysctl to 
 > sched_bind() for a user process as a hack to test it out.

You lost me at adding the sysctl for the user process..
Does FreeBSD have, or plan to have, an interface to bind threads to
CPUs?  

> 
 > John has a patch that pins interrupt threads, etc, not sure what the status of 
 > that is.  CC'd.

He wanted me to test it and I dropped the ball.  By the time I got a
chance (some months later), it was so stale it did not come close to
applying.  I've asked him to regen.

Drew