From nobody Sat Sep 16 20:06:21 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Rp2Bd5BnMz4tQB3 for ; Sat, 16 Sep 2023 20:06:33 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Rp2Bc0t8mz4Qkr for ; Sat, 16 Sep 2023 20:06:31 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of junchoon@dec.sakura.ne.jp has no SPF policy when checking 153.125.133.21) smtp.mailfrom=junchoon@dec.sakura.ne.jp; dmarc=none Received: from kalamity.joker.local (123-1-88-210.area1b.commufa.jp [123.1.88.210]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 38GK6LSF093728 for ; Sun, 17 Sep 2023 05:06:21 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Sun, 17 Sep 2023 05:06:21 +0900 From: Tomoaki AOKI To: freebsd-current@freebsd.org Subject: Re: CURRRENT snapshot won't boot due missing ZFS feature Message-Id: <20230917050621.1da562867be099a1674d785b@dec.sakura.ne.jp> In-Reply-To: <7EEF3435-064D-4C3C-98E4-2B27A788DB43@yahoo.com> References: <7EEF3435-064D-4C3C-98E4-2B27A788DB43.ref@yahoo.com> <7EEF3435-064D-4C3C-98E4-2B27A788DB43@yahoo.com> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: - X-Spamd-Result: default: False [-1.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; ONCE_RECEIVED(0.10)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; R_DKIM_NA(0.00)[]; HAS_ORG_HEADER(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; R_SPF_NA(0.00)[no SPF record]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DMARC_NA(0.00)[sakura.ne.jp]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4Rp2Bc0t8mz4Qkr On Sat, 16 Sep 2023 08:43:49 -0700 Mark Millard wrote: > void wrote on > Date: Sat, 16 Sep 2023 12:12:02 UTC : > > > On Sat, Sep 16, 2023 at 12:55:19PM +0100, Warner Losh wrote: > > > > >Yes. The boot loader comes from the host. It must know how to read ZFS. > > > > It knows how to read zfs. > > I expect Warner was indicating: you have a (efi?) loader that knows > how to deal with the features listed in: > > sys/contrib/openzfs/cmd/zpool/compatibility.d/openzfs-2.1-freebsd > > being active but not with some new feature(s) listed in: > > sys/contrib/openzfs/cmd/zpool/compatibility.d/openzfs-2.2 > > being active. > > The following are the "read-only-compatibile no" features > that are new in openzfs-2.2 compared to openzfs-2.1-freebsd : > > blake3 > ednor > head_errlog > vdev_zaps_v2 > > So any of those being active leads to lack of even read-only > activity being compatible. (Although, the loader's subset > of the potential overall activity might allow ignoring some > specific "read-only-compatibile no" status examples.) > > For reference: > > # diff -u99 /usr/main-src/sys/contrib/openzfs/cmd/zpool/compatibility.d/openzfs-2.1-freebsd /usr/main-src/sys/contrib/openzfs/cmd/zpool/compatibility.d/openzfs-2.2 > --- /usr/main-src/sys/contrib/openzfs/cmd/zpool/compatibility.d/openzfs-2.1-freebsd 2021-06-24 20:08:57.206621000 -0700 > +++ /usr/main-src/sys/contrib/openzfs/cmd/zpool/compatibility.d/openzfs-2.2 2023-06-10 15:59:25.354999000 -0700 > @@ -1,34 +1,40 @@ > -# Features supported by OpenZFS 2.1 on FreeBSD > +# Features supported by OpenZFS 2.2 on Linux and FreeBSD > allocation_classes > async_destroy > +blake3 > +block_cloning > bookmark_v2 > bookmark_written > bookmarks > device_rebuild > device_removal > draid > +edonr > embedded_data > empty_bpobj > enabled_txg > encryption > extensible_dataset > filesystem_limits > +head_errlog > hole_birth > large_blocks > large_dnode > livelist > log_spacemap > lz4_compress > multi_vdev_crash_dump > obsolete_counts > project_quota > redacted_datasets > redaction_bookmarks > resilver_defer > sha512 > skein > spacemap_histogram > spacemap_v2 > userobj_accounting > +vdev_zaps_v2 > +zilsaxattr > zpool_checkpoint > zstd_compress > > (Last I checked, /usr/share/zfs/compatibility.d/openzfs-2.2 does > not exist yet. Thus were I had the diff look.) It may be because it's not yet listed here, thus not installed. /usr/src/cddl/share/zfs/compatibility.d/Makefile > > On the host in question, there are many guests, > > some with zfs-boot, some not, just file-based. > > But with what openzfs features active vs. not active > in each case? > > > What the host is not, is zfs-on-root. It boots from ssd (ada0). > > The vdevs are on a sas disk array. > > > > >So either your bootable partitions must not have com.klarasystems:vdev_zaps_v2 > > >in your BEs or you must have a new user boot. I think you can just install > > >the one from 14, but haven't tried it. > > > > Can you briefly explain how I'd install the one from 14 please? > > > I do not use bhyve so I do not even know if the > context is using the efi loader from a msdosfs > vs. not. For efi loaders, copying from one msdosfs > with a sufficient vintage to the one with the wrong > vintage (replacing) is sufficient. > > For reference (from an aarch64 context): > > # find /boot/efi/EFI/ -print > /boot/efi/EFI/ > /boot/efi/EFI/FREEBSD > /boot/efi/EFI/FREEBSD/loader.efi > /boot/efi/EFI/BOOT > /boot/efi/EFI/BOOT/bootaa64.efi > > There may well be only: > > EFI/BOOT/bootaa64.efi > > for all I know. > > >From an amd64 context: > > # find /boot/efi/EFI/ -print > /boot/efi/EFI/ > /boot/efi/EFI/FREEBSD > /boot/efi/EFI/FREEBSD/loader.efi > /boot/efi/EFI/BOOT > /boot/efi/EFI/BOOT/bootx64.efi > > There may well be only: > > EFI/BOOT/bootx64.efi > > for all I know. > > (I set things up to have the EFI capitalization > so that referencing efi/ vs. EFI/ in my context > is unique for the mount point. vs. the msdosfs > directory.) > > === > Mark Millard > marklmi at yahoo.com -- Tomoaki AOKI