Possible bug in NFSv4 with krb5p security?

[Elias Mårtenson]

First of all, I used the "bug" word in the subject, and I'm not doing that
lightly. I fully understand that the initial reaction to such claim is "he
did something wrong", and frankly, that's what I'm hoping.

I've spent the last two weeks trying to get an NFS share working with krb5p
security from a FreeBSD server to OSX and Ubuntu clients. I've followed all
the documentation, read everything Google could find for me, asked on the
IRC channel and even asked on Stackexchange, all to no avail.

In all my reading, something struck me as odd: Nowhere did I find any
indication that anyone has actually set this up on 9.1-CURRENT. After
receiving zero replies on Stackexchange I started to think that perhaps
this is actually a bug.

Now, after all this talk, please let me explain what I've done. Most of
this text is taken verbatim from my Stackexchange question here:

http://serverfault.com/questions/477118/permissions-are-not-taking-effect-with-kerberised-nfsv4-on-freebsd

Problem summary
===============

My goal is to achieve the following:

 - Files served from the FreeBSD system
 - The only security model should be krb5p
 - Clients are Linux (Ubuntu) and OSX

The problem that I'm facing is that even though the Kerberos authentication
works, all accesses are performed using the user "nobody".

I can see the permissions when I do "ls -l". Even the user mapping works
correctly, but unless "nobody" has permission to do anything with the
files, I get a permission denied.

Here's an example interaction from the client (Ubuntu in this case, but the
same thing happens from OSX). In this example, /export/shared/testshare is
the shared directory from the FreeBSD server:

(I have changed the actual domain name to `domain` and the Kerberos realm
name to `REALM`)

    $ kinit
    Password for elias at REALM:
    $ klist
    Ticket cache: FILE:/tmp/krb5cc_1000_GBjtDP
    Default principal: elias at REALM

    Valid starting       Expires              Service principal
    09/02/2013 09:40:47  10/02/2013 09:40:44  krbtgt/REALM at REALM
    $ sudo mount -t nfs4 -osec=krb5p,vers=4 lion:/export/shared/testshare
/mnt
    $ ls -l /mnt
    total 4
    -rw-r--r-- 1 nobody nogroup   5 Feb  7 18:17 bar.txt
    -rw------- 1 elias  nogroup   4 Feb  5 23:09 foo.txt
    $ cat /mnt/bar.txt
    blah
    $ echo foo >>/mnt/bar.txt
    bash: /mnt/bar.txt: Permission denied
    $ cat /mnt/foo.txt
    cat: /mnt/foo.txt: Permission denied
    $ klist
    Ticket cache: FILE:/tmp/krb5cc_1000_GBjtDP
    Default principal: elias at REALM

    Valid starting       Expires              Service principal
    09/02/2013 09:40:47  10/02/2013 09:40:44  krbtgt/REALM at REALM
    09/02/2013 09:41:56  10/02/2013 09:40:44  nfs/lion.domain at REALM

Server configuration
====================

I have had quite some problems in finding a comprehensive guide to setting
up NFSv4 on FreeBSD. This is somewhat surprising in itself as I have found
that information on how to do things in FreeBSD to be very good.

Here are the relevant lines in /etc/rc.conf:

    rpcbind_enable="YES"
    nfs_server_enable="YES"
    nfsv4_server_enable="YES"
    nfsuserd_enable="YES"
    nfscbd_enable="YES"
    mountd_enable="YES"
    gssd_enable="YES"
    rpc_lockd_enable="YES"
    rpc_statd_enable="YES"
    zfs_enable="YES"

Here is the content of /etc/exports:

    /export/shared/testshare -sec=krb5p
    V4: / -sec=krb5p

Another interesting aspect is that when I used `tcpdump` to record the NFS
network traffic between the client and the server, I saw NFS3 packets
together with the NFS4 packets. Both of these packet types contained
encrypted data, so I still think Kerberos was used, but given the
configuration above, I would have expected there to be nothing but NFS4
traffic.

If anyone is able to confirm whether or not this actually has been tested
in 9.1-CURRENT, I'd appreciate it. Also, if not, then I'd love to know
where I should start looking for a solution. I'm experienced in system
level programming (having worked on Solaris at Sun in a previous life), but
a pointer where to start would be helpful.

Thanks and regards,
Elias