Side effect of net.inet.tcp.rexmit_min?

[Steve Francis]

On testing a 4.8 Free BSD system for short connections (basically a 
single HTTP get, then close of the connection), we found big differences 
from 4.6.

The 4.6 BSD system would comfortably do over 6000 connections per 
second- the 4.8 system was only doing about 4000/sec, and also 
fluctuating wildly in its rate, even though not CPU or NIC loaded.  (The 
4.8 is a faster machine with gig nics, cf the 4.6 which was 100Mbps.)

The behaviour of the 4.8 machine was made equal to the 4.6 machine (over 
6000 connections/sec, limited by our test bed) by changing 
net.inet.tcp.rexmit_min from default 1000 to 10.

However, all testing is on a LAN, and packet loss, AFAIK, is zero.
(Flood ping shows no loss; switch ports show no errors; netstat -sptcp 
show a very small number of retransmissions (See below); a tcptrace ran 
against tcpdump while net.inet.tcp.rexmit_min was 1000 reports no 
retranmissions.)

So why should changing net.inet.tcp.rexmit_min  have any impact on a 
lossless LAN?

Any help appreciated as to what further info I can collect to diagnose 
this..
Thanks

netstat -sptcp
tcp:
         205678066 packets sent
                 51416973 data packets (3553577611 bytes)
                 474 data packets (30344 bytes) retransmitted
                 0 resends initiated by MTU discovery
                 102843980 ack-only packets (75 delayed)
                 0 URG only packets
                 0 window probe packets
                 0 window update packets
                 51416642 control packets
         287817734 packets received
                 133565842 acks (for 3656410968 bytes)
                 14930 duplicate acks
                 0 acks for unsent data
                 102833595 packets (3826892727 bytes) received in-sequence
                 1177 completely duplicate packets (2370 bytes)
                 0 old duplicate packets
                 0 packets with some dup. data (0 bytes duped)
                 0 out-of-order packets (0 bytes)
                 51416434 packets (0 bytes) of data after window
                 0 window probes
                 51403368 window update packets
                 0 packets received after close
                 0 discarded for bad checksums
                 0 discarded for bad header offset fields
                 0 discarded because packet too short
         12 connection requests
         51416620 connection accepts
         0 bad connection attempts
         0 listen queue overflows
         51416629 connections established (including accepts)
         51424590 connections closed (including 0 drops)
                 5 connections updated cached RTT on close
                 5 connections updated cached RTT variance on close
                 0 connections updated cached ssthresh on close
         0 embryonic connections dropped
         133565841 segments updated rtt (of 51417199 attempts)
         9788 retransmit timeouts
                 0 connections dropped by rexmit timeout
         0 persist timeouts
                 0 connections dropped by persist timeout
         9 keepalive timeouts
                 9 keepalive probes sent
                 0 connections dropped by keepalive
         9 correct ACK header predictions
         51416725 correct data packet header predictions
         51416620 syncache entries added
                 191 retransmitted
                 122 dupsyn
                 0 dropped
                 51416620 completed
                 0 bucket overflow
                 0 cache overflow
                 0 reset
                 0 stale
                 0 aborted
                 0 badack
                 0 unreach
                 0 zone failures
         0 cookies sent
         0 cookies received