Re[5]: Assymetric NIC performance problem

Tue Oct 8 09:34:07 UTC 2013

On Mon, Oct  7, 2013 at 16:57 -07:00, Kevin Oberman <rkoberman at gmail.com> wrote:
>On Mon, Oct 7, 2013 at 8:44 AM, Konstantin Kuzvesov  < kuzvesov at list.ru > wrote:
>>
>>I've got a FreeBSD file server running Samba, file upload speeds are okay, but downloads are too slow.
>>First, I decided it is Samba's fault, but then I ran iperf tests...
>>
>>On a windows machine with gigabit NIC:
>>Z:\iperf>iperf -c 192.168.0.1
>>------------------------------------------------------------
>>Client connecting to 192.168.0.1, TCP port 5001
>>TCP window size: 64.0 KByte (default)
>>------------------------------------------------------------
>>[  3] local 192.168.0.2 port 1064 connected with 192.168.0.1 port 5001
>>[ ID] Interval       Transfer     Bandwidth
>>[  3]  0.0-10.2 sec  12.4 MBytes  10.2 Mbits/sec
>>
>>Z:\iperf>iperf -s
>>------------------------------------------------------------
>>Server listening on TCP port 5001
>>TCP window size: 64.0 KByte (default)
>>------------------------------------------------------------
>>[  4] local 192.168.0.2 port 5001 connected with 192.168.0.1 port 41220
>>[ ID] Interval       Transfer     Bandwidth
>>[  4]  0.0-10.0 sec   716 MBytes   600 Mbits/sec
>>
>>And on a another with FastEthernet NIC:
>>C:\iperf>iperf.exe -c 192.168.0.1
>>------------------------------------------------------------
>>Client connecting to 192.168.0.1, TCP port 5001
>>TCP window size: 64.0 KByte (default)
>>------------------------------------------------------------
>>[  3] local 192.168.0.5 port 4756 connected with 192.168.0.1 port 5001
>>[ ID] Interval       Transfer     Bandwidth
>>[  3]  0.0-10.1 sec  11.4 MBytes  9.42 Mbits/sec
>>
>>C:\iperf>iperf.exe -s
>>------------------------------------------------------------
>>Server listening on TCP port 5001
>>TCP window size: 64.0 KByte (default)
>>------------------------------------------------------------
>>[  4] local 192.168.0.5 port 5001 connected with 192.168.0.1 port 18558
>>[ ID] Interval       Transfer     Bandwidth
>>[  4]  0.0-10.0 sec   106 MBytes  88.5 Mbits/sec
>>
>>Both tests show server's NIC performance degradation to around 10Mbit/s when traffic goes from server to client. And everything works fine in other direction.
>>
>>I verified the cables and hub by simply connecting server and a test machine with a new short patch cord. I tried to change server's NIC from D-Link DGE-528T to Planet ENW-9604. And it became even worse -
>> using Planet NIC
>> speeds slowed down to around 500Mbit/s to server and the same 10Mbit/s to client. I tried to change NIC's media to 100BaseTX, it didn't help too. What else should I try to fix the problem? Maybe my system requires is just misconfigured?
>>
>>System configuration:
>>OS: FreeBSD 9.2-release
>>Kernel: generic
>>Firewall: none
>>/boot/loader.conf - load zfs modules only
>>/etc/sysctl.conf - empty
>>NIC: D-Link DGE-528T / Planet ENW-9604
>>  
>Output from ifconfig would probably be helpful, but I'd also 
suggest that you capture packets (or, at least headers) for at least the
 start of the transfer and look at them with wireshark or some similar 
tool. 
>
>wireshark can tell you quite a bit and, if you feed the 
capture into tcptrace, you can really see what is happening. (Note, 
wireshark provides indications of problems that can make sense to people
 not conversant with the gory details of TCP, though some issues may not 
be obvious. tcptrace output can be utterly cryptic if you don't 
understand a lot of the details of TCP and congestion control.
>
>Both
 wireshark and tcptrace are in ports and are best installed on a 
workstation. The tcpdump output can be used as input to both. ("tcpdump 
-pw FILE -i INTERFACE host ADDRESS" can do the job. Then copy the 
capture to the right place for analysis. But start with configuration 
and counters for the interface (netstat -i).
>-- 
>R. Kevin Oberman, Network Engineer
>E-mail:  rkoberman at gmail.com

Starting with configuration and counters.
First of all I ran the test again, since, imho, it's pointless to consider any counter on a just rebooted machine:
#iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[  4] local 192.168.0.1 port 5001 connected with 192.168.0.6 port 3892
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.3 sec  8.50 MBytes  6.91 Mbits/sec

#netstat -I re0
Name    Mtu Network       Address              Ipkts Ierrs Idrop    Opkts Oerrs  Coll
re0    1500 <Link#1>      xx:xx:xx:xx:xx:xx  3885118     0     0  6928524     0     0
re0    1500 192.168.0.0   ns.lan             3408770     -     -  6091524     -     -

#ifconfig re0
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
ether xx:xx:xx:xx:xx:xx
inet 192.168.0.1 netmask 0xffffff00 broadcast 192.168.0.255
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active

Then I did iperf test with UDP. Please, tell me if  this is normal. At the moment I don't know any reason for UDP to be so slooow.
#iperf -s -u
------------------------------------------------------------
Server listening on UDP port 5001
Receiving 1470 byte datagrams
UDP buffer size: 41.1 KByte (default)
------------------------------------------------------------
[  3] local 192.168.0.1 port 5001 connected with 192.168.0.6 port 3504
[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams
[  3]  0.0-10.0 sec  1.25 MBytes  1.05 Mbits/sec   1.112 ms    0/  892 (0%)
[  3]  0.0-10.0 sec  1 datagrams received out-of-order

-- 
Best regards,
Konstantin Kuzvesov