Re: block size: 512B configured, 4096B native all of a sudden

From: mike tancsa <mike_at_sentex.net>
Date: Mon, 22 Apr 2024 19:13:08 UTC
On 4/22/2024 12:05 PM, mike tancsa wrote:
> On 4/22/2024 11:49 AM, Dag-Erling Smørgrav wrote:
>> mike tancsa<mike@sentex.net>  writes:
>>> I was afraid of that. So basically I have to copy the entire pool or
>>> live with the performance penalty :(  73TB is a lot to copy / move.
>> Oh I didn't realize those disks were part of the larger pool, I thought
>> they were a separate pool named “special”.  You might be able to simply
>> detach and then re-attach both drives, afaik a draid can function
>> without its special vdev.
>
> Yeah, its an actual special vdev I added to speed up metadata and 
> small file access.  zpool-detach I think only works on mirrors and not 
> raidz vdevs.  From searching google, there does not seem to be a way 
> to detach a special vdev from a raidz pool.
>
>
I think I  confirmed my suspicion with a quick test pool

truncate -s 100G /quirk-test/junk.raw
truncate -s 100G /quirk-test/junk2.raw

mdconfig -f /quirk-test/junk.raw
mdconfig -f /quirk-test/junk2.raw
gpart create -s gpt md0
gpart create -s gpt md1
gpart add -s 20G -t freebsd-zfs /dev/md0
gpart add -s 20G -t freebsd-zfs /dev/md0
gpart add -s 20G -t freebsd-zfs /dev/md0
gpart add -s 20G -t freebsd-zfs /dev/md1
gpart add -s 20G -t freebsd-zfs /dev/md1
gpart add -s 20G -t freebsd-zfs /dev/md1
zpool create testpool raidz1 /dev/md0p1 /dev/md1p1 /dev/md0p2 /dev/md1p2
zpool add testpool special mirror /dev/md0p3 /dev/md1p3
zpool status


   pool: testpool
  state: ONLINE
config:

         NAME        STATE     READ WRITE CKSUM
         testpool    ONLINE       0     0     0
           raidz1-0  ONLINE       0     0     0
             md0p1   ONLINE       0     0     0
             md1p1   ONLINE       0     0     0
             md0p2   ONLINE       0     0     0
             md1p2   ONLINE       0     0     0
         special
           mirror-1  ONLINE       0     0     0
             md0p3   ONLINE       0     0     0
             md1p3   ONLINE       0     0     0


  # zpool remove testpool mirror-1
cannot remove mirror-1: invalid config; all top-level vdevs must have 
the same sector size and not be raidz.

zpool detach just nukes the mirror I think for the special vdev

#  zpool detach testpool /dev/md1p3

# zpool status
   pool: quirk-test
  state: ONLINE
   scan: resilvered 128G in 00:08:52 with 0 errors on Mon Apr  8 
14:37:29 2024
config:

         NAME        STATE     READ WRITE CKSUM
         quirk-test  ONLINE       0     0     0
           raidz1-0  ONLINE       0     0     0
             da4p1   ONLINE       0     0     0
             da1p1   ONLINE       0     0     0
             da2p1   ONLINE       0     0     0
             da5p1   ONLINE       0     0     0

errors: No known data errors

   pool: testpool
  state: ONLINE
config:

         NAME        STATE     READ WRITE CKSUM
         testpool    ONLINE       0     0     0
           raidz1-0  ONLINE       0     0     0
             md0p1   ONLINE       0     0     0
             md1p1   ONLINE       0     0     0
             md0p2   ONLINE       0     0     0
             md1p2   ONLINE       0     0     0
         special
           md0p3     ONLINE       0     0     0

errors: No known data errors

  # zpool remove testpool /dev/md0p3
cannot remove /dev/md0p3: invalid config; all top-level vdevs must have 
the same sector size and not be raidz.

  # zpool detach testpool /dev/md0p3
cannot detach /dev/md0p3: only applicable to mirror and replacing vdevs


     ---Mike