NFS client slow on amd64 6.2-PRERELEASE #2

[Heinrich Rebehn]

Hi list,

despite recent improvements with the nfs code, client performance still
seems to be a problem. I am getting < 2 MB/sec where i would expect at
least 10 MB/sec.

My Setup:

Machine  | ANTSRV1            | ANTSRV2
==================================================
Board    | Tyan 2882-D        | ASUS A7V8X-X
CPU      | Opteron 246        | Athlon XP1800+
Mem	 | 2G                 | 1G
NIC      | Broadcom BCM5704C  | Intel PRO/1000MT

OS:6.2-PRERELEASE #2 as of 01-OCT-2006
64Bit OS on ANTSRV1, 32Bit on ANTSRV2

The 2 machines are connected via a 1GB switch

============================
Trying UDP first
============================
root at antsrv1 [/tmp] # mount_nfs -U -L antsrv2:/export/disk2 /mnt
root at antsrv1 [/tmp] # rm /mnt/x/*
root at antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/10MB.dat bs=1M count=10
10+0 records in
10+0 records out
10485760 bytes transferred in 4.967248 secs (2110980 bytes/sec)

real    0m5.253s
user    0m0.005s
sys     0m0.022s
root at antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/100MB.dat bs=1M
count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 69.020366 secs (1519227 bytes/sec)

real    1m9.276s
user    0m0.028s
sys     0m0.211s

==============================
Now TCP
==============================
root at antsrv1 [/tmp] # umount /mnt
root at antsrv1 [/tmp] # mount_nfs -T -L antsrv2:/export/disk2 /mnt
root at antsrv1 [/tmp] # rm /mnt/x/*
root at antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/10MB.dat bs=1M count=10
10+0 records in
10+0 records out
10485760 bytes transferred in 5.289492 secs (1982376 bytes/sec)

real    0m5.312s
user    0m0.001s
sys     0m0.023s
root at antsrv1 [/tmp] # time dd if=/dev/zero of=/mnt/x/100MB.dat bs=1M
count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 58.715595 secs (1785856 bytes/sec)

real    0m58.740s
user    0m0.000s
sys     0m0.204s

===============================
SCP
===============================
root at antsrv1 [/tmp] # scp 100MB.dat antsrv2:/tmp/x
Password:
100MB.dat                                            100%  100MB
11.1MB/s   00:09

===============================
Iperf
===============================
root at antsrv1 [~] # iperf -c antsrv2
------------------------------------------------------------
Client connecting to antsrv2, TCP port 5001
TCP window size: 32.5 KByte (default)
------------------------------------------------------------
[  3] local 134.102.176.16 port 59123 connected with 134.102.176.17 port
5001
[  3]  0.0-10.0 sec    743 MBytes    623 Mbits/sec

NFS from a Linux machine (Kernel 2.6.17.8) (connected via 10Mb LAN)
yields a constant value of ~10MB/sec (TCP and UDP), so the FreeBSD NFS
server seems to be ok.

=================================
Reversing the setup
=================================
root at antsrv2 [~] # mount_nfs -L -U antsrv1:/export/huge/x /mnt
root at antsrv2 [~] # time dd if=/dev/zero of=/mnt/10MB.dat bs=1M count=10
10+0 records in
10+0 records out
10485760 bytes transferred in 0.533470 secs (19655758 bytes/sec)

real    0m0.547s
user    0m0.001s
sys     0m0.085s
root at antsrv2 [~] # time dd if=/dev/zero of=/mnt/100MB.dat bs=1M count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 7.647475 secs (13711401 bytes/sec)

real    0m7.673s
user    0m0.000s
sys     0m0.920s
root at antsrv2 [~] # umount /mnt
root at antsrv2 [~] # mount_nfs -L -T antsrv1:/export/huge/x /mnt
root at antsrv2 [~] # rm /mnt/*
root at antsrv2 [~] # time dd if=/dev/zero of=/mnt/10MB.dat bs=1M count=10
10+0 records in
10+0 records out
10485760 bytes transferred in 0.524917 secs (19976030 bytes/sec)

real    0m0.557s
user    0m0.000s
sys     0m0.085s
root at antsrv2 [~] # time dd if=/dev/zero of=/mnt/100MB.dat bs=1M count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 5.723110 secs (18321787 bytes/sec)

real    0m5.756s
user    0m0.000s
sys     0m0.704s


This yields much better results. Could this be a 64bit-specific problem?

Both machines have statd and lockd running.

Any ideas where i could tune?

-- 

Heinrich Rebehn

University of Bremen
Physics / Electrical and Electronics Engineering
- Department of Telecommunications -

Phone : +49/421/218-4664
Fax   :            -3341
_______________________________________________
freebsd-stable at freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"