[rfc] 64-bit inode numbers

[Rick Macklem]

John Baldwin wrote:
> On Friday, June 24, 2011 11:38:35 pm Benjamin Kaduk wrote:
> > > point. fts(3) and friends will assume that it is a mount point
> > > crossing when st_dev changes. It will then expect that the funny
> > > rule that the d_ino in dirent will not be the same as st_ino.
> > >
> > > What I do for NFSv4 is sythesize the mnt_stat.f_fsid value and
> > > return that as st_dev for the mounted volume until I see the fsid
> > > returned by the server change. Below that point, I return the fsid
> > > from the server as st_dev so long as it isn't the same as the
> >
> > I think I'm confused. You're ... walking a directory heirarchy, and
> > return a fake st_dev value but hold onto the fsid value from the
> > server,
> > then when the fsid from the server changes (due to a ... different
> > NFS
> > mount?), start reporting that new fsid and throw away the fake
> > st_dev
> > value? Can you point me at the code that is doing this?
> 
> I think he's saying that VOP_GETATTR() for different vnodes in a
> single NFSv4
> "mount" (as in 'struct mount *') can return different st_dev values to
> userland where the st_dev value for a given vnode depends on the
> remote
> fsid of the file on the NFSv4 server. That is, for NFSv4 it seems that
> all
> files on a mount do not use the same value of st_dev (as they would
> for a
> local filesystem), but instead only files from the logical volume on
> the
> server share an st_dev. That is, st_dev is per-vnode rather than just
> copied
> from the mount. This is done by storing va_fsid in the NFS attribute
> cache
> for each vnode:
> 
> int
> nfscl_loadattrcache(struct vnode **vpp, struct nfsvattr *nap, void
> *nvaper,
> void *stuff, int writeattr, int dontshrink)
> {
> ...
> /*
> * For NFSv4, if the node's fsid is not equal to the mount point's
> * fsid, return the low order 32bits of the node's fsid. This
> * allows getcwd(3) to work. There is a chance that the fsid might
> * be the same as a local fs, but since this is in an NFS mount
> * point, I don't think that will cause any problems?
> */
> if (NFSHASNFSV4(nmp) && NFSHASHASSETFSID(nmp) &&
> (nmp->nm_fsid[0] != np->n_vattr.na_filesid[0] ||
> nmp->nm_fsid[1] != np->n_vattr.na_filesid[1])) {
> /*
> * va_fsid needs to be set to some value derived from
> * np->n_vattr.na_filesid that is not equal
> * vp->v_mount->mnt_stat.f_fsid[0], so that it changes
> * from the value used for the top level server volume
> * in the mounted subtree.
> */
> if (vp->v_mount->mnt_stat.f_fsid.val[0] !=
> (uint32_t)np->n_vattr.na_filesid[0])
> vap->va_fsid = (uint32_t)np->n_vattr.na_filesid[0];
> else
> vap->va_fsid = (uint32_t)hash32_buf(
> np->n_vattr.na_filesid, 2 * sizeof(uint64_t), 0);
> } else
> vap->va_fsid = vp->v_mount->mnt_stat.f_fsid.val[0];
> ...
> }
> 
> Then for VOP_GETATTR() it returns the va_fsid from the attribute cache
> saved in 'vap' as the vnode's va_fsid which is used to compute st_dev
> in
> vn_stat().
> 
> I think the effect here is that 'mount' still only shows a single
> mountpoint
> for NFSv4, but applications that check for 'st_dev' changing to see if
> they
> are crossing a mountpoint (e.g. find -x) will treat the volumes as
> different
> mountpoints.
> 
Yes, John. You said it way better than I did:-)

This is necessary for NFSv4 because the server crosses server mount
points (unlike NFSv3 where servers do not) and, as such, st_ino is
not unique within one client NFSv4 mount (struct mount *).

Without this, things like "ls -R" will complain about cycles when
the same <st_dev, st_ino> tuple is seen again.

rick