ZFS feature compatibility?

[Toomas Soome]

> On 25. Jan 2021, at 22:15, mike tancsa <mike at sentex.net> wrote:
> 
> On 1/25/2021 2:37 PM, Toomas Soome via freebsd-current wrote:
>> 
>>> On 25. Jan 2021, at 21:31, Michael Butler via freebsd-current <freebsd-current at freebsd.org> wrote:
>>> 
>>> I have a few machines on which I've been hesitant to run 'zpool upgrade' as I'm not sure of the (boot?) implications. They report like this ..
>>> 
>>> imb at toshi:/home/imb> uname -a
>>> FreeBSD toshi.auburn.protected-networks.net 14.0-CURRENT FreeBSD 14.0-CURRENT #25 main-eb61de5b78: Fri Jan 22 10:03:02 EST 2021 root at toshi.auburn.protected-networks.net:/usr/obj/usr/src/amd64.amd64/sys/TOSHI amd64
>>> 
>>> imb at toshi:/home/imb> zpool status
>>> pool: zroot
>>> state: ONLINE
>>> status: Some supported features are not enabled on the pool. The pool can
>>>       still be used, but some features are unavailable.
>>> action: Enable all features using 'zpool upgrade'. Once this is done,
>>>       the pool may no longer be accessible by software that does not support
>>>       the features. See zpool-features(5) for details.
>>> 
>>> Is it safe to upgrade the root pool?
>>> 
>>> 	imb
>> We can not boot from encrypted pool and draid. Rest is all ok. Please note, you may need to update the bootblocks.
>> 
> last Friday on zoo.freebsd.org <http://zoo.freebsd.org/>, mjg at freebsd.org <mailto:mjg at freebsd.org> and I could not boot
> again because v2 bookmarks were on the boot pool.  I had to boot from
> another disk, remove the bookmarks and then boot. This was on RELENG_13
> (stable/13-c256203-g51d73a3e46c)
> 
>     —Mike

/*
 * List of ZFS features supported for read
 */
static const char *features_for_read[] = {
        "org.illumos:lz4_compress",
        "com.delphix:hole_birth",
        "com.delphix:extensible_dataset",
        "com.delphix:embedded_data",
        "org.open-zfs:large_blocks",
        "org.illumos:sha512",
        "org.illumos:skein",
        "org.zfsonlinux:large_dnode",
        "com.joyent:multi_vdev_crash_dump",
        "com.delphix:spacemap_histogram",
        "com.delphix:zpool_checkpoint",
        "com.delphix:spacemap_v2",
        "com.datto:encryption",
        "com.datto:bookmark_v2",
        "org.zfsonlinux:allocation_classes",
        "com.datto:resilver_defer",
        "com.delphix:device_removal",
        "com.delphix:obsolete_counts",
        "com.intel:allocation_classes",
        "org.freebsd:zstd_compress",
        "com.delphix:bookmark_written",
        NULL
};

Are you sure you have bootblocks updated? ESP for UEFI boot and freebsd-boot for BIOS boot.

rgds,
toomas