New optimized soreceive_stream() for TCP sockets, proof of concept

[Andrew Gallatin]

John Baldwin writes:
 > > John has a patch that pins interrupt threads, etc, not sure what the status 
 > of 
 > > that is.  CC'd.
 > 
 > Tested and around for over a year.  Sent to people several times but no 
 > benchmarking has resulted.  It lives in p4 in //depot/user/jhb/intr/...
 > 
 > I've just regenerated the patch at 
 > http://www.FreeBSD.org/~jhb/patches/intr_bind.patch (same URL as the last N 

It seems very useful, in conjunction with some sort of CPU binding
API.  With ULE, and netserver bound via a hack to CPU 1, I
can nearly double the bandwidth by binding my ithread
to the opposite CPU using your patch.

BTW, you need a little sanity checking somehere in the API
I tried to remove binding, and guessed at binding to -1 rather
than reading the source.  That resulted in this panic:

# ./ibind/ibind 256 -1
kernel trap 12 with interrupts disabled


Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address   = 0x3806778bc
fault code              = supervisor read data, page not present
instruction pointer     = 0x8:0xffffffff8040a509
stack pointer           = 0x10:0xffffffff91e88ad0
frame pointer           = 0x10:0xffffffff91e88b00
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = resume, IOPL = 0
current process         = 788 (ibind)
[thread pid 788 tid 100047 ]
Stopped at      intr_bind+0xe9: movl    cpu_apic_ids(,%rdi,4),%esi
db> bt
Tracing pid 788 tid 100047 td 0xffffff001ea1b840
intr_bind() at intr_bind+0xe9
sysarch() at sysarch+0x14a
ia32_syscall() at ia32_syscall+0x236
Xint0x80_syscall() at Xint0x80_syscall+0x60

Thanks again!

Drew