Rewritten TCP reassembly

[Andrew Gallatin]

Andre Oppermann writes:

 > 
 > I have already the next round in the works which is optimized even more
 > by merging consecutive mbuf chains together (at the moment I have packet
 > segment chains which have a direct pointer to the mbuf at the end of the
 > chain) and which get passed in one go to soappend_stream.  This removes
 > the "present" loop and simplifies the general code a bit more again.

Great..  I've been a little busy, and have only run tests -- I haven't
even looked at the code ;)

 > With this and two other optimizations I have in mind you should be able
 > to get very close to the theoretical maximum bandwidth of your current
 > 4Gig Myrinet cards.
 > 
 > There are a couple of other TCP tweaks that would help your special case
 > some more now though.

FWIW, the out-of-order frames are a firmware bug that we hope to fix
soon.  Its just icing on the cake that the bug makes such a nice test
case for you ;)

With no copy overhead (kttcp), 3.95Gb/sec is easily achievable,
even in 5-stable with the old TCP reassembly code.  

Drew