ZFS raidz device replacement problem

[Pawel Jakub Dawidek]

On Fri, Apr 13, 2007 at 11:06:16PM -0500, Barry Pederson wrote:
> I've been playing with ZFS (awesome stuff, thanks PJD) and noticed something funny when replacing a device under a raidz pool.  It seems that even though ZFS says 
> resilvering is complete, you still need to manually do a "zpool scrub" to really get the pool into a good state.

How do you tell it's not in a good state?

> From what I've read in the "Solaris ZFS Administration Guide", it doesn't seem that that step should be required.  Is there some kind of auto-scrub being missed?
> 
> I've tried to show this below with some md devices, creating 4 of them, putting 3 into a raidz and then replacing one.  This is with a world and kernel csupped and built 
> earlier today (2007-04-13)
[...]
> # zpool create mypool raidz md0 md1 md2
[...]
> # zpool replace mypool md2 md3
[...]
> # zpool status mypool
>   pool: mypool
>  state: ONLINE
>  scrub: resilver completed with 0 errors on Fri Apr 13 22:43:19 2007
> config:
> 
>         NAME           STATE     READ WRITE CKSUM
>         mypool         ONLINE       0     0     0
>           raidz1       ONLINE       0     0     0
>             md0        ONLINE       0     0     0
>             md1        ONLINE       0     0     0
>             replacing  ONLINE       0     0     0
>               md2      ONLINE       0     0     0
>               md3      ONLINE       0     0     0

Do you mean that resilver is completed, but it is still replacing? ZFS
resilvers only live data, those md's was free, so it completed
immediately.

> # zpool status mypool
>   pool: mypool
>  state: ONLINE
>  scrub: resilver completed with 0 errors on Fri Apr 13 22:43:19 2007
> config:
> 
>         NAME        STATE     READ WRITE CKSUM
>         mypool      ONLINE       0     0     0
>           raidz1    ONLINE       0     0     0
>             md0     ONLINE       0     0     0
>             md1     ONLINE       0     0     0
>             md3     ONLINE       0     0     0

It looks everything is fine. What's wrong?

> # zpool scrub mypool
> 
> # zpool status mypool
>   pool: mypool
>  state: ONLINE
> status: One or more devices has experienced an unrecoverable error.  An
>         attempt was made to correct the error.  Applications are unaffected.
> action: Determine if the device needs to be replaced, and clear the errors
>         using 'zpool clear' or replace the device with 'zpool replace'.
>    see: http://www.sun.com/msg/ZFS-8000-9P
>  scrub: scrub completed with 0 errors on Fri Apr 13 22:43:46 2007
> config:
> 
>         NAME        STATE     READ WRITE CKSUM
>         mypool      ONLINE       0     0     0
>           raidz1    ONLINE       0     0     0
>             md0     ONLINE       0     0     0
>             md1     ONLINE       0     0     0
>             md3     ONLINE       0     0     5

If you are referring to this CKSUM count not beeing 0, this was a bug in
ZFS itself, and was fixes in OpenSolaris already and fix was merged to
FreeBSD.

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20070415/6551e183/attachment.pgp