From nobody Sun Aug 28 01:25:35 2022 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MFbVk2Jv3z4ZPZp for ; Sun, 28 Aug 2022 01:25:50 +0000 (UTC) (envelope-from jo@bruelltuete.com) Received: from email.jo-t.de (seppel.jo-t.de [45.132.244.126]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4MFbVh6JySz3Qmt; Sun, 28 Aug 2022 01:25:48 +0000 (UTC) (envelope-from jo@bruelltuete.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=bruelltuete.com; s=bruelltuete18a; t=1661649941; bh=znItd5sobhVri4Fkns/2ceLF1vGxQSmC5M/iDghAxNo=; h=Message-ID:Date:MIME-Version:Subject:To:References:From:From; b=oe31WrsDEqy2zNMQZymj4kQx/xvBcRarBpSDDwo0firpLLpqODnf3018/4MbKvk4X daa4ueUA8lUDkT/m4VJ0FQLYEaO/N8z8KvOL2mTu517CUy0HrTFBh4AXiJisMHHcrZ p2pqS+pb6Fk8eal2htMJ9i3JaUi3gyyeU7ppMfi+6+u3yuOIt7rM0YgyfH2ZVdnscH FqBLkPDzcoVHCUSTD057ybKLdXLDF70Rr+jkfo5F+5yXQZTNVEAdO2B8EiSEjsbUCb kOdyfxoxbj/XwMNQVVloIVNlE6WjXGUUUvT7QDMwKG7BFrgbYkNHK5VGDpg4Ahjou2 eyw2ez5Bn0n9A== Message-ID: Date: Sun, 28 Aug 2022 02:25:35 +0100 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Subject: Re: ZFS bootfs vs vfs.root.mountfrom To: Warner Losh , Peter Jeremy , Kyle Evans Cc: FreeBSD FS Newsgroups: gmane.os.freebsd.devel.file-systems References: Content-Language: en-GB From: Johannes Totz In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4MFbVh6JySz3Qmt X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bruelltuete.com header.s=bruelltuete18a header.b=oe31WrsD; dmarc=pass (policy=none) header.from=bruelltuete.com; spf=pass (mx1.freebsd.org: domain of jo@bruelltuete.com designates 45.132.244.126 as permitted sender) smtp.mailfrom=jo@bruelltuete.com X-Spamd-Result: default: False [-3.99 / 15.00]; NEURAL_HAM_LONG(-1.00)[-0.998]; NEURAL_HAM_MEDIUM(-1.00)[-0.998]; NEURAL_HAM_SHORT(-0.99)[-0.994]; DMARC_POLICY_ALLOW(-0.50)[bruelltuete.com,none]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_ALLOW(-0.20)[bruelltuete.com:s=bruelltuete18a]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; TO_DN_ALL(0.00)[]; ASN(0.00)[asn:197540, ipnet:45.132.244.0/22, country:DE]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[bruelltuete.com:+]; RCPT_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On 27/08/2022 05:15, Warner Losh wrote: > > > On Fri, Aug 26, 2022 at 10:01 PM Peter Jeremy > wrote: > > When booting from ZFS was first introduced, the recommended approach > was to configure the root filesystem via vfs.root.mountfrom.  See (e.g.) > https://lists.freebsd.org/pipermail/freebsd-fs/2011-September/012482.html > https://lists.freebsd.org/pipermail/svn-src-head/2011-October/030641.html > and people wanted to change that to use the zpool bootfs property - e.g. > https://lists.freebsd.org/pipermail/freebsd-current/2009-October/012933.html > https://lists.freebsd.org/pipermail/freebsd-fs/2010-March/008010.html > resulting in it becoming optional in SVN r235330 in May 2012: > https://lists.freebsd.org/pipermail/svn-src-head/2012-May/036902.html > > > Yes. This was a total hack at the time. The boot loader normally sets > this variable > w/o users intervening and this hack propagated into loader.conf files, > which has > always been a bad idea. > > The recommendation to use vfs.root.mountfrom remains in parts of the > wiki (e.g. https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/Mirror > step > 2.6) and on mailing list posts (e.g. the thread starting at > https://lists.freebsd.org/pipermail/freebsd-fs/2020-July/028351.html > ). > The loader(8) and loader.conf(5) man pages state that the root > filesystem is set by vfs.root.mountfrom, with a default derived from > /etc/fstab or, for ZFS only, currdev (the boot filesystem). > > > Yes. That's how it is supposed to work. The boot loader sets this based on > where it loaded the kernel. This variable should *NEVER* be set in a > loader.conf file unless you have a 'very special snowflake' system that > can't determine it automatically. ZFS root isn't special. > > OTOH, https://wiki.freebsd.org/BootEnvironments > states "/etc/fstab > must be purged of any / mount, and /boot/loader.conf must not be > setting vfs.root.mountfrom directly."  I can't find any other > reference to this requirement. > > > /etc/fstab isn't used for zfs root. I believe that BootEnvironments is 100% > right here. > > As for the BootEnvironment management tools themselves: > * Neither beadm(8) nor bectl(8) mention vfs.root.mountfrom. > > > This is correct. > > * beadm(8) refers to > https://forums.freebsd.org/showthread.php?t=31662 > , >   which recommends setting vfs.root.mountfrom > > > This is bad advice. > > * beadm(8) will update vfs.root.mountfrom in /boot/loader.conf if it >   exists but will not add it if it doesn't exist. > > > This is terrible. It's part of the continuing papering over of bugs that > have > long ago been fixed and should, at most, warn that's its set and that it > really shouldn't be. > > * bectl(8) ignores /boot/loader.conf > > > This is proper behavior. > > The last point means that switching an existing system from beadm to > bectl is very likely to result in the system mounting the wrong root > filesystem: An existing system is likely to have vfs.root.mountfrom > set, since that used to be the only approach and is still widely > recommended, whilst bectl ignores it and will therefore leave the > old vfs.root.mountfrom value present. > > > Yes. bectl likely should, at most, warn that it's set in loader.conf as an > antifootshooting measure, but only due to the all-too-long history. > > IMO, if setting vfs.root.mountfrom in /boot/loader.conf is no longer > supported for ZFS, we need to do a better job of documenting that and > removing documentation to the contrary.  I also believe that bectl > should at least warn if /boot/loader.conf sets vfs.root.mountfrom. > > > It really never had been the proper way. One should almost never do > it today, except in the above 'special snowflake' systems that still crop up > from time to time (we have one at work since we still have some systems > that use gmirror roots, and the boot loader doesn't set it right for > them since > we can't for logistical reasons set /etc/fstab, but that's not ZFS > related). We > should actively militate against it with extreme prejudice, though, for > routine > uses. We should rip out all references in the docs related to ZFS. We should > warn if we detect it today. I agree completely with what you are saying > here. > I would be extraordinarily surprised to find anybody with a contrary > opinion. Isn't a legitimate use-case where /boot and / are on different devices/pools? That's what I'm using it for. bootfs property says where to find the kernel, and the accompanying vfs.root.mountfrom in loader.conf says where the matching rootfs is (in my case on a different pool). > > Now, who is going to do the work? :) > > Warner