ZFS and FAULTED devices (corrupted data), can't make the pool ONLINE again

[Attila Nagy]

Hello,

FreeBSD RELENG_7, x86, a terrible disk array, called Promise RM-8000 
with 8 disks on an ahc.
The pool is a RAIDZ2.
Tomorrow the array went crazy (its firmware is a total crap), so I had 
to reboot both the machine and the disk array.

The effect:
  pool: people
 state: DEGRADED
status: One or more devices could not be used because the label is 
missing or
        invalid.  Sufficient replicas exist for the pool to continue
        functioning in a degraded state.
action: Replace the device using 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-4J
 scrub: resilver completed with 0 errors on Thu Nov 22 10:45:27 2007
config:

        NAME        STATE     READ WRITE CKSUM
        people      DEGRADED     0     0     0
          raidz2    DEGRADED     0     0     0
            da8     ONLINE       0     0     0
            da3     FAULTED      0     0     0  corrupted data
            da5     ONLINE       0     0     0
            da6     ONLINE       0     0     0
            da7     ONLINE       0     0     0
            da9     ONLINE       0     0     0
            da10    ONLINE       0     0     0
            da4     FAULTED      0     0     0  corrupted data

errors: No known data errors

I've tried everything I could think of:
zpool replace people da3
invalid vdev specification
use '-f' to override the following errors:
da3 is in use (r1w1e1)

zpool replace -f people da3
invalid vdev specification
the following errors must be manually repaired:
da3 is in use (r1w1e1)

zpool offline people da3
cannot offline da3: no valid replicas

zpool online people da3
Bringing device da3 online
(nothing has changed, the device is still FAULTED)

Hmm. Is this this bug? 
http://www.opensolaris.org/jive/thread.jspa?messageID=161812

-- 
Attila Nagy                                   e-mail: Attila.Nagy at fsn.hu
Free Software Network (FSN.HU)                 phone: +3630 306 6758
http://www.fsn.hu/