From nobody Mon Dec 26 19:32:05 2022 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Ngnwv5gNMz1HV1y for ; Mon, 26 Dec 2022 19:32:15 +0000 (UTC) (envelope-from list_freebsd@bluerosetech.com) Received: from echo.brtsvcs.net (echo.brtsvcs.net [IPv6:2607:f740:c::4ae]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Ngnwt6tjdz3s85 for ; Mon, 26 Dec 2022 19:32:14 +0000 (UTC) (envelope-from list_freebsd@bluerosetech.com) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of list_freebsd@bluerosetech.com designates 2607:f740:c::4ae as permitted sender) smtp.mailfrom=list_freebsd@bluerosetech.com; dmarc=none Received: from chombo.houseloki.net (65-100-43-2.dia.static.qwest.net [65.100.43.2]) by echo.brtsvcs.net (Postfix) with ESMTPS id 5E8CE38D07 for ; Mon, 26 Dec 2022 19:32:07 +0000 (UTC) Received: from [10.26.25.100] (ivy.pas.ds.pilgrimaccounting.com [10.26.25.100]) by chombo.houseloki.net (Postfix) with ESMTPSA id EFEB112991 for ; Mon, 26 Dec 2022 11:32:06 -0800 (PST) To: freebsd-fs@freebsd.org From: Mel Pilgrim Subject: zpool bootfs restrictions when boot and root pools are different? Message-ID: Date: Mon, 26 Dec 2022 11:32:05 -0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.12.0 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-1.60 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.30)[-0.295]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; ASN(0.00)[asn:36236, ipnet:2607:f740:c::/48, country:US]; DMARC_NA(0.00)[bluerosetech.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4Ngnwt6tjdz3s85 X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N I asked this on freebsd-questions a while back and didn't get an answer, so I'm trying here instead. The setup: "bootkey" pool: A UEFI-boot USB flashdrive with /boot on it. It's ZFS so I can use copies=2 and scrubs to ensure integrity, but it was UFS in its prior incarnation. "nvme" pool: Contains the BootEnv-managed root. The nvme pool sits on GELI-encrypted drives, and the keys are on the bootkey pool. This is so the keys are never on the same devices as the partitions they decrypt. This is where I run into an issue: BootEnv uses the bootfs property to steer the loader, but I can't set bootfs to point to a different pool: # zpool set bootfs=nvme/ROOT/13.1p5 bootkey cannot set property for 'bootkey': 'nvme/ROOT/13.1p5' is an invalid name In order for BootEnv to "work" I let it set bootfs on nvme instead: # zpool get bootfs NAME PROPERTY VALUE SOURCE bootkey bootfs - default mech bootfs - default nvme bootfs nvme/ROOT/13.1p5 local and have a script that copies that value to /boot/loader.conf: # grep ROOT /boot/loader.conf vfs.root.mountfrom="zfs:nvme/ROOT/13.1p5" That somewhat defeats the ease of BootEnv, but does make the rest of it work. With neither vfs.root.mountfrom nor bootfs set on bootkey, the kernel assumes /boot is on the root filesystem then panics because it can't find /sbin/init. And now the questions: 1. Why does the bootfs property restrict the value to only datasets on that pool if it also requires the pool name in the value? 2. Is there a way around this that would make BootEnv work without vfs.root.mountfrom in loader.conf? 3. If the answer to (2) is no, would it be welcome if I added to bectl the option to have it use vfs.root.mountfrom instead of bootfs?