NFSv3 + krb5 mysteries - need help tracking down

[Attila Bogár]

Hi,

I have a FreeBSD 9-STABLE acting as a kerberized NFSv3 server.

server# ktutil list
FILE:/etc/krb5.keytab:

Vno  Type                     Principal
   5  aes256-cts-hmac-sha1-96 nfs/server.linguamatics.com at LINGUAMATICS.COM
   5  des3-cbc-sha1 nfs/server.linguamatics.com at LINGUAMATICS.COM
   5  arcfour-hmac-md5 nfs/server.linguamatics.com at LINGUAMATICS.COM

ntp in sync everywhere

The network is a lagg device composed of two bce interfaces (an add-in 
card).

-- 8< [nfstest.sh] --
#!/bin/bash

i=0
fail=0
while [ $i -lt 100 ]
do
   i=$[i+1]
   echo "RUN: $i"
   umount -f /mnt
   sleep 1
   mount -v -o sec=krb5i,vers=3,proto=tcp server:/export/share /mnt || fail
=$[fail+1]
done
echo "$fail times failed"
-- 8< --

centos62# ./nfstest.sh
54 times failed

ubuntu1204# ./nfstest.sh
98 times failed

ubuntu1104# ./nfstest.sh
0 times failed

centos58# ./nfstest.sh
0 times failed

I started rpc.gssd -vvvvv on all linux clients.

The clients which did not fail are using gssapi v1 with DES.
Jun 29 18:17:41 centos58 rpc.gssd[1452]: prepare_krb5_rfc1964_buffer: 
serializing keys with enctype 4 and length 8
Jun 29 18:04:36 ubuntu1104 rpc.gssd[911]: prepare_krb5_rfc1964_buffer: 
serializing keys with enctype 4 and length 8
The failing clients are using the newer gssapi v2 with AES256.
Jun 29 17:59:37 ubuntu1204 rpc.gssd[756]: prepare_krb5_rfc4121_buffer: 
serializing key with enctype 18 and size 32
Jun 29 17:55:48 centos62 rpc.gssd[1183]: prepare_krb5_rfc4121_buffer: 
serializing key with enctype 18 and size 32

Note the different RFC being used.  This is just a suspicion, this may 
not be realted to the problem.
The cipher being used is different too.

Then I changed my script to proto=udp.
from ubuntu1104 fails 0 times.
from centos62 fails 0 times.

On centos58 and ubuntu1204 mount locks up all the time.

Then I added to krb5.conf [libdefaults]
default_tgs_enctypes = dec-cbc-crc and rebooted both centos58 and 
ubuntu1204.

After rebooting centos56 and ubuntu1204:

nfstest fails 0 times on centos58 with udp
I get very long response times for ubuntu1204 mounts and always a 
permission denied.

This is a mystery.

I have not tested NFSv4 yet.

I need some help to track down this problem.

Attila

PS: This may be the same problem as this thread:
http://lists.freebsd.org/pipermail/freebsd-stable/2012-June/068619.html