9.0-RELEASE-p4 + NFS + ZFS = issues... :/ (probably a memory leak)

[Xavier Beaudouin]

Hello there,

I try to make a FreeBSD ZFS NFS server with the following configuration 
:

- Supermicro X7DB8
- 2 L5420
- 12G RAM
- 8 x 300G SCSI drives using the 2 U320 scsi channel.
- lagg (LACP) of 2 e1000
- zpool :

# zpool status
   pool: vol
  state: ONLINE
  scan: scrub repaired 0 in 2h11m with 0 errors on Thu Nov  1 07:42:05 
2012
config:

         NAME        STATE     READ WRITE CKSUM
         vol         ONLINE       0     0     0
           raidz1-0  ONLINE       0     0     0
             da0     ONLINE       0     0     0
             da4     ONLINE       0     0     0
           raidz1-1  ONLINE       0     0     0
             da1     ONLINE       0     0     0
             da5     ONLINE       0     0     0
           raidz1-2  ONLINE       0     0     0
             da2     ONLINE       0     0     0
             da6     ONLINE       0     0     0
           raidz1-3  ONLINE       0     0     0
             da3     ONLINE       0     0     0
             da7     ONLINE       0     0     0


I have tried this patch : http://people.freebsd.org/~namei-leak.patch

And issue still here.

Also tried to move from newnfs to oldnfs server and client, same issue.

Issue is comming when copying from a netapp to this machine a 70G data 
with bunch of big files (eg 600M ~ 8G) and web files.

All NFS clients use NFSv3 and UDP.

Tried :

nfs client netapp and freebsd on the same machine : 1srt rsync from 
netapp to freebsd ok : no issue, second rsync : memory leak, swap is 
used (about less than 100Mb, but this trigger the bug.

nfs client of the netapp is the "filer" machine, and the destination is 
local zfs spool. The rsync to make up to date data trigger as well the 
issue.

/boot/loader.conf
zfs_load="YES"
if_lagg_load="YES"
autoboot_delay="10"             # Delay in seconds before autobooting,
start
# http://icesquare.com/wordpress/how-to-improve-zfs-performance/
#I have 8G of Ram
#vfs.zfs.prefetch_disable=0

#If Ram = 4GB, set the value to 512M
#If Ram = 8GB, set the value to 1024M
#vfs.zfs.arc_min="1024M"

#Ram x 0.5 - 512 MB
#vfs.zfs.arc_max="3584M"

#Ram x 2
vm.kmem_size_max="24G"

#Ram x 1.5
#vm.kmem_size="12G"


/etc/sysctl.conf
# $FreeBSD: release/9.0.0/etc/sysctl.conf 112200 2003-03-13 18:43:50Z 
mux $
#
#  This file is read when going to multi-user and its contents piped 
thru
#  ``sysctl'' to adjust kernel values.  ``man 5 sysctl.conf'' for 
details.
#

# Uncomment this to prevent users from seeing information about 
processes that
# are being run under another UID.
#security.bsd.see_other_uids=0
kern.maxfiles=64000
vfs.nfsd.maxthreads=16
vfs.zfs.prefetch_disable=1


The latter (vfs.zfs.prefetch_disable=1) make the bug a bit strange. 
Inactive memory keeps growing, but active and free is still very low 
during the copy. Then machine use all of his memory, and NFS server not 
responsive.

/etc/rc.conf
hostname="filernew"
keymap="fr.iso.acc.kbd"
cloned_interfaces="lagg0"
ifconfig_em0="up"
ifconfig_em1="up"
ifconfig_lagg0="inet 172.31.x.x netmask 255.255.0.0 laggproto lacp 
laggport em0 laggport em1"
defaultrouter="172.31.x.x"
sshd_enable="YES"
ntpd_enable="YES"
powerd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="NO"
zfs_enable="YES"
puppet_enable="YES"
# NFS
rpcbind_enable="YES"
nfs_server_enable="YES"
nfs_server_flags="-u -t -n 4"
mountd_flags="-r"
mountd_enable="YES"
rpc_lockd_enable="YES"
rpc_statd_enable="YES"

# Added by Puppet
bsnmpd_enable="YES"

Kind regards...

Xavier

-- 
Xavier Beaudouin