Extended attributes for NFSv3 - possible Linux interop

[Robert N. M. Watson]

On 2 Jul 2010, at 01:44, James Morris wrote:

> I've been working on an implementation of extended attributes for the 
> Linux NFSv3 code, initially based on the IRIX implementation [1].  This is 
> being developed as a sideband protocol ('XATTR'), to convey simple 
> name/value string extended attributes over the network.
> 
> Here's a link to the latest patchset:
> http://lwn.net/Articles/392944/
> 
> and my LinuxCon slides on the topic from last year:
> http://namei.org/presentations/linuxcon09_nfsv3xattrs.pdf
> 
> The code is at a working prototype stage, and the next step is to begin 
> formalizing the protocol.
> 
> I'd like to try and determine whether there is any interest in this XATTR 
> protocol for FreeBSD (which has a similar extended attribute model to 
> Linux), and then whether we should develop an informal specification to 
> cater for interoperability between FreeBSD and Linux. 
> 
> [1] http://oss.sgi.com/projects/ob1.nuked/src/nfs/

Hi Other Robert:

I think it's a good idea. BTW, Mac OS X *also* supports meta-data extended attributes (not just forks) on HFS+ and friends, and similarly uses them to hold ACLs, MAC labels (using a port of FreeBSD's MAC Framework), etc. My hazy recollection is that they are stored inside a specially designated file fork, but that might be a misrecollection. So the Apple folk might be interested as well so we should try to get them in the loop -- they certainly also understand the file fork vs. meta-data EA tension. Their API is a bit more like the Linux API than the FreeBSD API when it comes to meta-data -- minor spelling differences. Historically other systems have also done IRIX meta-data EAs (which is where both the Linux and FreeBSD work found its model) including HPFS on OS 2. On a similar note, we should engage the NetApp folk and Sun folk; I'm suspect NetApp does not support our notion of meta-data EAs with atomicity properties, and I'm pretty sure Sun doesn't. However, if there will be standards, getting their buy-in early makes sense.

FreeBSD currently doesn't implement Sun's NFSv3 ACL extension, which I believe is what Linux implements, but probably should. If we're going to standardize the EA RPC interface, we should also standardize the ACL interface, and then someone in FreeBSD land can implement it as well :-). And that probably means extending the existing ACL extension: we will need to support NFSv4 ACLs with NFSv3, since we can use both POSIX and NFSv4 ACLs on disk in FreeBSD, and the leaning is towards NFSv4 ACLs for everything these days. Mac OS X uses NTFS ACLs, which are (in many ways) similar to NFSv4 ACLs and also might be interested.

Something that's always worried me about the Mac OS X and Linux EA APIs is the namespace issue: in FreeBSD, we have an explicit enumeration of namespaces reflected in an argument to the system calls -- in our case, an int, but you could imagine other options. As I recall, Linux (perhaps also IRIX?) designated "system" EAs via an interpretation of names ('$' prefix). Mac OS X doesn't do this, or at least, the namespace is different. I wonder given the portability concerns: would it make sense to make the NFS wire protocol identify the namespace explicitly, and then OSes can map to/from their local implementation semantics? Linux could strip the '$' before putting the name on the wire and select the system namespace in the RPC, whereas FreeBSD could map its local notion of EXTATTR_NAMESPACE_SYSTEM into whatever the NFS constant is. Then when it pops out the other end, mapping back to local semantics would occur. This seems more likely to get the security right (in FreeBSD, writing to the system namespace requires a specific privilege in our privilege system), and is an adequate superset of all the APIs to be useful.

On a related note, it might make sense to review current consumers of the EA API and decide if we want to standardize a system call API as well -- current API consumers have a bit of a hard time right now due to the differences across all major UNIX platforms, which is silly. Especially if we have an NFS EA RPC in flight, taking this opportunity to nail down a new, portable, system call API would be opportune. Obviously, we'd need to think about the namespace issue, but also atomicity issues: one of the guarantees we provide in FreeBSD is that EA writes are atomic with respect to other simultaneous users, and we also try really hard to provide strong atomicity properties with respect to persistence (i.e., in the event of a crash you will get $oldlabel for MAC, or $newlabel, but not half of each). Further, we offer transactional semantics in our VFS layer, but that isn't exposed to userspace. This allows atomic update of several EAs at once as required by our MAC environment (if you run with two policies and both update labels simultaneously, it will be atomic). Something to think about offering via system calls, I suppose, but I worry that pushes the edge a bit far on UNIX APIs. :-)

I'm happy to help contribute to the writing on an Internet Draft and/or RFC -- the lack of NFS support for EAs (and the EA vs. file fork confusion) have long caused me frustration, and with systems like SELinux, our various MAC policies, and all sorts of other things floating around, there's a strong motivation to fix this ... in a portable way! I'm just sorry I haven't gotten to this sooner...

I've added Rick Macklem to the CC line since he did both our original NFS code, and our more recent NFSv4 code, and is a long-time participant in the NFS community.

Could we set up a phone call to chat about all this? Things are really busy here this summer, both for good and bad reasons, but a phone call on East Coast time is usually arrangeable for me (I'm mostly on British time). Any chance you might be at USENIX Security in DC this August?

Another Robert