TX performance problems with 3Com 905C cards

[Marian Durkovic]

Hi all,

  I'm experiencing some performance issues with 3Com 905C cards @ FreeBSD 4.8
When running nttcp tests with other servers, the result is:

otherbox -> box with 3c905

     Bytes  Real s   CPU s Real-MBit/s  CPU-MBit/s   Calls  Real-C/s   CPU-C/s
l409600000   34.80    2.96     94.1489   1105.1982  100000   2873.20   33728.0
1409600000   34.82    1.18     94.1175   2773.2828  281323   8080.27  238094.6

box with 3c905 -> otherbox

     Bytes  Real s   CPU s Real-MBit/s  CPU-MBit/s   Calls  Real-C/s   CPU-C/s
l409600000   38.86    1.18     84.3323   2766.3223  282788   7277.88  238733.7
1409600000   38.85    1.56     84.3524   2100.2557  100000   2574.23   64094.7

The above data shows that 3c905C is able to receive data at full wire speed 
of FastEthernet since the value corresponds to its theoretical maximum.
However, when 3c905C is transmitting data, the rate drops to 84 Mbps.

TCPdump doesn't show any packet retransmissions or other problems, however,
when 3c905C is sending data, the delay betwen the time when data packet is
sent out and it's corresponding ACK increases significantly.

With 128 KB TCPwindow, the ACKs are delivered 12 msec after the packet.
With 256 KB TCPwindow, the ACKs are delivered 24 msec after the packet.

I.e. regardless of TCPwindow size, the rate is limited to 84 Mbps.

It looks like the ACKs are delivered to the kernel somehow delayed, in fact
only when TCP stops sending data because the full TCPwindow of unacknowledged
data is in transit. Seems like something works in half-duplex mode, although
the card is connected to 100 Mbps fullduplex port and both the switch and
ifconfig -a confirm 100baseTX full duplex. 

Is this a known hardware limitation of these cards? Is there something
that needs to be tuned on the PCI bus? Or is there some problem in the xl 
driver, causing such delays or e.g. assuming half-duplex operation?


    Thanks in advance for your help.

	M.

--------------------------------------------------------------------------
----                                                                  ----
----   Marian Durkovic                       network  manager         ----
----                                                                  ----
----   Slovak Technical University           Tel: +421 2 524 51 301   ----
----   Computer Centre, Nam. Slobody 17      Fax: +421 2 524 94 351   ----
----   812 43 Bratislava, Slovak Republic    E-mail/sip: md at bts.sk    ----
----                                                                  ----
--------------------------------------------------------------------------