zfs detach/replace

[Michael DeMan]

Hi,

Sounds realistic to me that the 'replace' command works after a 'detach' is properly done.

In my case, I forgot to do the 'detach' - just did the 'replace' after changing the drive (and after a reboot).  From there, ZFS gets in the state below - where it automagically created label/ada5LABEL/old and appears to be wanting to do the rebuild onto label/ada5LABEL as if it is part of a mirrored pair?

This pool was built on FreeBSD 8.0, with an operating system update to 8.1 after that.  Possibly I could try and update the ZFS pool version or something if this is fixed now?  

I know for a fact, back with FreeBSD 7.x, that this same scenario could occur.  Basically on below - my old notes show there is no way to fix the situation with ada5LABEL and ada5LABEL/old without destroying and rebuilding the pool.  Any attempts to 'detach', 'offline' or anything else on either of those two logical entries fails with a 'no valid replicas'.


$ zpool status
  pool: zp1rz2
 state: DEGRADED
status: One or more devices has experienced an unrecoverable error.  An
	attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
	using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: scrub in progress for 53h42m, 80.53% done, 12h59m to go
config:

	NAME                       STATE     READ WRITE CKSUM
	zp1rz2                     DEGRADED     0     0     0
	  raidz2                   DEGRADED     0     0     0
	    label/ada0LABEL        ONLINE       0     0     0
	    label/ada1LABEL        ONLINE       0     0     0
	    label/ada2LABEL        ONLINE      35     0     0  19.5K repaired
	    label/ada3LABEL        ONLINE       0     0     0
	    label/ada4LABEL        ONLINE       0     0     0
	    replacing              UNAVAIL      0 3.24M     0  insufficient replicas
	      label/ada5LABEL/old  UNAVAIL      0 3.74M     0  cannot open
	      label/ada5LABEL      UNAVAIL      0 3.74M     0  cannot open
	    label/ada6LABEL        ONLINE       0     0     0
	    label/ada7LABEL        ONLINE       0     0     0

errors: No known data errors



- Mike

On Jan 1, 2012, at 11:54 AM, Jeremy Chadwick wrote:

> On Sun, Jan 01, 2012 at 11:03:21AM -0800, Michael DeMan wrote:
>> Right, I botched it and did a 'replace' on a drive that is in an
>> raidz2 pool.  replace is for replacing a mirrored drive.
> 
> I just want to clarify for readers: this statement is inaccurate.
> 
> "zpool replace" can (should) be used on anything (mirrors, raidzX,
> etc.).
> 
> I have used "zpool replace" myself on FreeBSD on raidz1 without problems
> for years.  Here's valid proof, and this is not "simulated" or done on a
> VM or anything like that -- this is bare metal.
> 
> http://koitsu.wordpress.com/2010/07/22/freebsd-and-zfs-hot-swapping-sata-disks-with-ahci/
> 
> Full commands/output are provided there as well (there's a link within
> my blog post providing full output).
> 
> -- 
> | Jeremy Chadwick                                jdc at parodius.com |
> | Parodius Networking                       http://www.parodius.com/ |
> | UNIX Systems Administrator                   Mountain View, CA, US |
> | Making life hard for others since 1977.               PGP 4BD6C0CB |
>