ZFS: drive replacement performance

[Gabor Radnai]

> On Tue, Jul 7, 2009 at 3:26 PM, Mahlon E. Smith <mahlon at martini.nu <http://lists.freebsd.org/mailman/listinfo/freebsd-stable>> wrote:

*> > On Tue, Jul 07, 2009, Freddie Cash wrote:
*> >* >
*> >* > This is why we've started using glabel(8) to label our drives, and then
*> >* add
*> >* > the labels to the pool:
*> >* >   # zpool create store raidz1 label/disk01 label/disk02 label/disk03
*> >* >
*> >* > That way, it does matter where the kernel detects the drives or what the
*> >* > physical device node is called, GEOM picks up the label, and
ZFS uses the
*> >* > label.
*> >*
*> >* Ah, slick.  I'll definitely be doing that moving forward.  Wonder if I
*> >* could do it piecemeal now via a shell game, labeling and replacing each
*> >* individual drive?  Will put that on my "try it" list.
*> >*
*
> Yes, this can be done piecemeal, after the fact, on an already configured
> pool.  That's how I did it on one of our servers.  It was originally

> configured using the device node names (da0, da1, etc).  Then I set up the
> second server, but used labels.  Then I went back to the first server,
> labelled the drives, and did "zpool replace storage da0 label/disk01" for

> each drive.  Doesn't take long to resilver, as it knows that it's the same
> device.

It seems a very good practice but how did you do actually on your
first already configured server?
I am struggling with the followings:

1. on online disk, member of configured raidz pool, "glabel label"
fails with error message "operation not permitted".
2. if I make disk offline, glabel succeed, but making disk back to
online clears label.

3. if I make disk offline, glabel succeed, but "zpool replace <pool>
<dev> <label>" fails with "/dev/label/<label> is part of
 active pool <pool>"
4. export <pool>, glabel, import <pool> neither works.

A detailed guide in "for dummies" style would be appreciated.

Thanks,
Gabor