Re: ZFS bootfs vs vfs.root.mountfrom

On Fri, Aug 26, 2022 at 10:01 PM Peter Jeremy <peterj@freebsd.org> wrote:

> When booting from ZFS was first introduced, the recommended approach
> was to configure the root filesystem via vfs.root.mountfrom.  See (e.g.)
>  https://lists.freebsd.org/pipermail/freebsd-fs/2011-September/012482.html
>  https://lists.freebsd.org/pipermail/svn-src-head/2011-October/030641.html
> and people wanted to change that to use the zpool bootfs property - e.g.
>
> https://lists.freebsd.org/pipermail/freebsd-current/2009-October/012933.html
>  https://lists.freebsd.org/pipermail/freebsd-fs/2010-March/008010.html
> resulting in it becoming optional in SVN r235330 in May 2012:
>  https://lists.freebsd.org/pipermail/svn-src-head/2012-May/036902.html


Yes. This was a total hack at the time. The boot loader normally sets this
variable
w/o users intervening and this hack propagated into loader.conf files,
which has
always been a bad idea.

The recommendation to use vfs.root.mountfrom remains in parts of the
> wiki (e.g. https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/Mirror step
> 2.6) and on mailing list posts (e.g. the thread starting at
> https://lists.freebsd.org/pipermail/freebsd-fs/2020-July/028351.html).
> The loader(8) and loader.conf(5) man pages state that the root
> filesystem is set by vfs.root.mountfrom, with a default derived from
> /etc/fstab or, for ZFS only, currdev (the boot filesystem).
>

Yes. That's how it is supposed to work. The boot loader sets this based on
where it loaded the kernel. This variable should *NEVER* be set in a
loader.conf file unless you have a 'very special snowflake' system that
can't determine it automatically. ZFS root isn't special.


> OTOH, https://wiki.freebsd.org/BootEnvironments states "/etc/fstab
> must be purged of any / mount, and /boot/loader.conf must not be
> setting vfs.root.mountfrom directly."  I can't find any other
> reference to this requirement.
>

/etc/fstab isn't used for zfs root. I believe that BootEnvironments is 100%
right here.


> As for the BootEnvironment management tools themselves:
> * Neither beadm(8) nor bectl(8) mention vfs.root.mountfrom.
>

This is correct.


> * beadm(8) refers to https://forums.freebsd.org/showthread.php?t=31662,
>   which recommends setting vfs.root.mountfrom
>

This is bad advice.


> * beadm(8) will update vfs.root.mountfrom in /boot/loader.conf if it
>   exists but will not add it if it doesn't exist.
>

This is terrible. It's part of the continuing papering over of bugs that
have
long ago been fixed and should, at most, warn that's its set and that it
really shouldn't be.


> * bectl(8) ignores /boot/loader.conf
>

This is proper behavior.


> The last point means that switching an existing system from beadm to
> bectl is very likely to result in the system mounting the wrong root
> filesystem: An existing system is likely to have vfs.root.mountfrom
> set, since that used to be the only approach and is still widely
> recommended, whilst bectl ignores it and will therefore leave the
> old vfs.root.mountfrom value present.
>

Yes. bectl likely should, at most, warn that it's set in loader.conf as an
antifootshooting measure, but only due to the all-too-long history.


> IMO, if setting vfs.root.mountfrom in /boot/loader.conf is no longer
> supported for ZFS, we need to do a better job of documenting that and
> removing documentation to the contrary.  I also believe that bectl
> should at least warn if /boot/loader.conf sets vfs.root.mountfrom.
>

It really never had been the proper way. One should almost never do
it today, except in the above 'special snowflake' systems that still crop up
from time to time (we have one at work since we still have some systems
that use gmirror roots, and the boot loader doesn't set it right for them
since
we can't for logistical reasons set /etc/fstab, but that's not ZFS
related). We
should actively militate against it with extreme prejudice, though, for
routine
uses. We should rip out all references in the docs related to ZFS. We should
warn if we detect it today. I agree completely with what you are saying
here.
I would be extraordinarily surprised to find anybody with a contrary
opinion.

Now, who is going to do the work? :)

Warner