Re: vfs.zfs.bclone_enabled (was: FreeBSD 14.0-BETA2 Now Available) [block_cloning and zilsaxattr missing from loader's features_for_read]

On Sep 18, 2023, at 15:51, Mark Millard <marklmi@yahoo.com> wrote:

> Alexander Motin <mav_at_FreeBSD.org> wrote on
> Date: Mon, 18 Sep 2023 13:26:56 UTC :
> 
>> block_cloning feature is marked as READONLY_COMPAT. It should not 
>> require any special handling from the boot code.
> 
> From stand/libsa/zfs/zfsimpl.c but adding a comment about the
> read-only compatibility status of each entry:
> 
> /*
> * List of ZFS features supported for read
> */
> static const char *features_e: vfs.zfs.bclone_enabled

Sorry, a stupid error of mine messed up the above line.
It should have been:

static const char *features_for_read[] = {

>        "com.datto:bookmark_v2", // READ-ONLY COMPATIBLE no
>        "com.datto:encryption", // READ-ONLY COMPATIBLE no
>        "com.datto:resilver_defer", // READ-ONLY COMPATIBLE yes
>        "com.delphix:bookmark_written", // READ-ONLY COMPATIBLE no
>        "com.delphix:device_removal", // READ-ONLY COMPATIBLE no
>        "com.delphix:embedded_data", // READ-ONLY COMPATIBLE no
>        "com.delphix:extensible_dataset", // READ-ONLY COMPATIBLE no
>        "com.delphix:head_errlog", // READ-ONLY COMPATIBLE no
>        "com.delphix:hole_birth", // READ-ONLY COMPATIBLE no
>        "com.delphix:obsolete_counts", // READ-ONLY COMPATIBLE yes
>        "com.delphix:spacemap_histogram", // READ-ONLY COMPATIBLE yes
>        "com.delphix:spacemap_v2", // READ-ONLY COMPATIBLE yes
>        "com.delphix:zpool_checkpoint", // READ-ONLY COMPATIBLE yes
>        "com.intel:allocation_classes", // READ-ONLY COMPATIBLE yes
>        "com.joyent:multi_vdev_crash_dump", // READ-ONLY COMPATIBLE no
>        "com.klarasystems:vdev_zaps_v2", // READ-ONLY COMPATIBLE no
>        "org.freebsd:zstd_compress", // READ-ONLY COMPATIBLE no
>        "org.illumos:lz4_compress", // READ-ONLY COMPATIBLE no
>        "org.illumos:sha512", // READ-ONLY COMPATIBLE no
>        "org.illumos:skein", // READ-ONLY COMPATIBLE no
>        "org.open-zfs:large_blocks", // READ-ONLY COMPATIBLE no
>        "org.openzfs:blake3", // READ-ONLY COMPATIBLE no
>        "org.zfsonlinux:allocation_classes", // READ-ONLY COMPATIBLE yes
>        "org.zfsonlinux:large_dnode", // READ-ONLY COMPATIBLE no
>        NULL
> };
> 
> So it appears that the design is that both "no" and "yes" ones
> that are known to be supported are listed and anything else is
> supposed to lead to rejection until explicitly added as
> known-compatibile.
> 
> This matches up with stand/libsa/zfs/zfsimpl.c 's:
> 
> static int
> nvlist_check_features_for_read(nvlist_t *nvl)
> {
>        nvlist_t *features = NULL;
>        nvs_data_t *data;
>        nvp_header_t *nvp;
>        nv_string_t *nvp_name;
>        int rc;
> 
>        rc = nvlist_find(nvl, ZPOOL_CONFIG_FEATURES_FOR_READ,
>            DATA_TYPE_NVLIST, NULL, &features, NULL);
>        switch (rc) {
>        case 0:
>                break;          /* Continue with checks */
> 
>        case ENOENT:
>                return (0);     /* All features are disabled */
> 
>        default:
>                return (rc);    /* Error while reading nvlist */
>        }
> 
>        data = (nvs_data_t *)features->nv_data;
>        nvp = &data->nvl_pair;  /* first pair in nvlist */
> 
>        while (nvp->encoded_size != 0 && nvp->decoded_size != 0) {
>                int i, found;
> 
>                nvp_name = (nv_string_t *)((uintptr_t)nvp + sizeof(*nvp));
>                found = 0;
> 
>                for (i = 0; features_for_read[i] != NULL; i++) {
>                        if (memcmp(nvp_name->nv_data, features_for_read[i],
>                            nvp_name->nv_size) == 0) {
>                                found = 1;
>                                break;
>                        }
>                }
> 
>                if (!found) {
>                        printf("ZFS: unsupported feature: %.*s\n",
>                            nvp_name->nv_size, nvp_name->nv_data);
>                        rc = EIO;
>                }
>                nvp = (nvp_header_t *)((uint8_t *)nvp + nvp->encoded_size);
>        }
>        nvlist_destroy(features);
> 
>        return (rc);
> }
> 
> I do not know if vfs.zfs.bclone_enabled=0 leads the loader
> to see vs. not-see a "com.fudosecurity:block_cloning".

It appears that 2 additions afeter opebzfas-2.1-freebsd are
missing from the above list:

com.fudosecurity:block_cloning
org.openzfs:zilsaxattr

See, for reference (but shorter naming):

# diff -u99 /usr/main-src/sys/contrib/openzfs/cmd/zpool/compatibility.d/openzfs-2.1-freebsd /usr/main-src/sys/contrib/openzfs/cmd/zpool/compatibility.d/openzfs-2.2
--- /usr/main-src/sys/contrib/openzfs/cmd/zpool/compatibility.d/openzfs-2.1-freebsd 2021-06-24 20:08:57.206621000 -0700
+++ /usr/main-src/sys/contrib/openzfs/cmd/zpool/compatibility.d/openzfs-2.2 2023-06-10 15:51:13.220607000 -0700
@@ -1,34 +1,40 @@
-# Features supported by OpenZFS 2.1 on FreeBSD
+# Features supported by OpenZFS 2.2 on Linux and FreeBSD
 allocation_classes
 async_destroy
+blake3
+block_cloning
 bookmark_v2
 bookmark_written
 bookmarks
 device_rebuild
 device_removal
 draid
+edonr
 embedded_data
 empty_bpobj
 enabled_txg
 encryption
 extensible_dataset
 filesystem_limits
+head_errlog
 hole_birth
 large_blocks
 large_dnode
 livelist
 log_spacemap
 lz4_compress
 multi_vdev_crash_dump
 obsolete_counts
 project_quota
 redacted_datasets
 redaction_bookmarks
 resilver_defer
 sha512
 skein
 spacemap_histogram
 spacemap_v2
 userobj_accounting
+vdev_zaps_v2
+zilsaxattr
 zpool_checkpoint
 zstd_compress

( openzfs-2.2 is not available from the standard place yet. )

The other "+" lines are in the list already.

===
Mark Millard
marklmi at yahoo.com