Exposing a hierarchy of ZFS datasets inside multiple jails

[Paul Schenkeveld]

Hi,

On Fri, Jun 17, 2011 at 02:46:59PM -0400, Lars Kellogg-Stedman wrote:
> Hello all,
> 
> Hi there,
> 
> I am trying to expose a hierarchy of home directories to a number of
> FreeBSD jails. The home directories are configured such that each is a
> unique ZFS dataset. The jails are used for development work and hence
> are created and destroyed on a regular basis.
> 
> My first thought was simply to use nullfs to mount /home inside the
> jail, but nullfs doesn't provide any way to access subordinate
> filesystems.
> 
> My second thought was to export the directories via NFS and then run
> the automounter daemon (amd) inside each jail. This would have Just
> Worked...if it were possible to perform NFS mounts inside a jail. But
> it's not.
> 
> My third thought was to run amd on the host and provision nullfs
> mounts into the jails...but amd support for nullfs doesn't exist.
> 
> My fourth thought was to go back to exporting the directories using
> NFS, because of course amd works with NFS, right? Unfortunately,
> rather than mounting a directory on the target mountpoint, amd likes
> to mount things in a temporary location (/.amd_mnt/...) and then
> create a symlink...which, of course, is useless inside the jail
> environment.t
> 
> So maybe you could use nullfs to expose a subdirectory of /.amd_mnt to
> the jail? No! This brings us back to my first attempt, in which we
> find that there is no way to access subordinate filesystems using
> nullfs.
> 
> And then my head exploded.
> 
> Is there a good solution for what I'm trying to do? A bad solution
> would be to run a script after booting the jail that would create
> multiple nullfs mountpoints for all the home directories, but this is
> pretty clunky -- it would need to be run periodically to take into
> account new directories or removed directories. So basically I would
> have to write a poorly designed automounter.
> 
> There must be a better way. How are other folks solving this?
> 
> It looks like there are discussions going back several years about
> setting the VFCF_JAIL on NFS filesystems, but it these haven't
> resulted in any changes to the released code.  Is this the best way to
> go?  In theory, if I build a kernel under which NFS is jail friendly I
> can go ahead and run amd inside the jail

Probably not a good solution but to stir the pool of thoughts a bit...

Nullfs mounts and NFS mounts operate on filesystems (or datasets) and
do not include subordinates.  Smbfs operates on directory (sub)trees
so have /home and /home/user[123...] datasets outside the jails, run
samba there with a share called [home] (not to be confused with the
[homes] share that comes with smb.conf.sample) and mount this share
using mount_smbfs inside every jail (from fstab.<jailname>).

Just my $.02

Regards,

Paul Schenkeveld