ZFS pool faulted (corrupt metadata) but the disk data appears ok...

[Ben RUBSON]

On 02 Feb 2018 11:26, Michelle Sullivan wrote:

Hi Michelle,

> Michelle Sullivan wrote:
>> Michelle Sullivan wrote:
>>> So far (few hours in) zfs import -fFX has not faulted with this image...
>>> it's running out of memory currently about 16G of 32G- however 9.2-P15
>>> kernel died within minutes... out of memory (all 32G and swap) so am
>>> more optimistic at the moment...  Fingers Crossed.
>>>
>> And the answer:
>>
>> 11-STABLE on a USB stick.
>>
>> Remove the drive that was replacing the hotspare (ie the replacement
>> drive for the one that initially died)
>> zpool import -fFX storage
>> zpool export storage
>>
>> reboot back to 9.x
>> zpool import storage
>> re-insert drive replacement drive.
>> reboot
> Gotta thank people for this again, saved me again this time on a  
> non-FreeBSD system this time (with a lot of using a modified recoverdisk  
> for OSX - thanks PSK@)... Lost 3 disks out of a raidz2 and 2 more had  
> read errors on some sectors.. don't know how much (if any) data I've lost  
> but at least it's not a rebuild from back up of all 48TB..

What about the root-cause ?
Sounds like you had 5 disks dying at the same time ?
Do you periodically run long smart tests ? Zpool scrubs ?

Thx,

Ben