Re: ZFS: Rescue FAULTED Pool

Hi,

> On 7 Feb 2025, at 08:48, A FreeBSD User <freebsd@walstatt-de.de> wrote:
> 
> Am Tue, 4 Feb 2025 12:18:06 +0200
> Andriy Gapon <avg@FreeBSD.org> schrieb:
> 
>> On 01/02/2025 10:57, A FreeBSD User wrote:
>>> Hello, this exactly happens when trying to import the pool. Prior to the loss, device da1p1
>>> has been faulted with numbers in the colum/columns "corrupted data"/further not seen now.
>>> 
>>> 
>>>  ~# zpool import
>>>    pool: BUNKER00
>>>      id: XXXXXXXXXXXXXXXXXXXX
>>>   state: FAULTED
>>> status: The pool metadata is corrupted.
>>>  action: The pool cannot be imported due to damaged devices or data.
>>>         The pool may be active on another system, but can be imported using
>>>         the '-f' flag.
>>>    see:https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-72
>>>  config:
>>> 
>>>         BUNKER00    FAULTED  corrupted data
>>>           raidz1-0  ONLINE
>>>             da2p1   ONLINE
>>>             da3p1   ONLINE
>>>             da4p1   ONLINE
>>>             da7p1   ONLINE
>>>             da6p1   ONLINE
>>>             da1p1   ONLINE
>>>             da5p1   ONLINE
>>> 
>>> 
>>>  ~# zpool import -f BUNKER00
>>> cannot import 'BUNKER00': I/O error
>>>         Destroy and re-create the pool from
>>>         a backup source.
>>> 
>>> 
>>> ~# zpool import -F BUNKER00
>>> cannot import 'BUNKER00': one or more devices is currently unavailable  
>> 
>> 
>> Too late now, but another useful command for situations like this is
>> zdb -G BUNKER00
>> It would print a log of various pool import actions.
> 
> Not "too late", very much appreciated, such helpful worst-case-scenario would have a nice
> place in a small section "desaster recovery" in the handbook, wouldn't it? The handbook is
> quite superficial at that point ...

+1

> Kind regards,
> Oliver
>> 
>> E.g., on a good pool:
>> # zdb -G rpool
>> 
>> ZFS_DBGMSG(zdb) START:
>> spa.c:5694:spa_open_common(): spa_open_common: opening rpool
>> spa_misc.c:419:spa_load_note(): spa_load(rpool, config trusted): LOADING
>> vdev.c:162:vdev_dbgmsg(): disk vdev '/dev/gpt/S6PEN.rpool': best uberblock found 
>> for spa rpool. txg 61892397
>> spa_misc.c:419:spa_load_note(): spa_load(rpool, config untrusted): using 
>> uberblock with txg=61892397
>> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading checkpoint txg
>> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading indirect 
>> vdev metadata
>> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Checking feature flags
>> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading special 
>> MOS directories
>> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading properties
>> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading AUX vdevs
>> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading vdev metadata
>> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading dedup tables
>> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading BRT
>> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Verifying Log Devices
>> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Verifying pool data
>> spa_misc.c:419:spa_load_note(): spa_load(rpool, config trusted): spa_load_verify 
>> found 0 metadata errors and 4 data errors
>> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Calculating 
>> deflated space
>> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Starting import
>> spa.c:8925:spa_async_request(): spa=rpool async request task=2048
>> spa_misc.c:419:spa_load_note(): spa_load(rpool, config trusted): LOADED
>> ZFS_DBGMSG(zdb) END
>> 
>> On a bad pool, the log may have helped to identify the exact problem.
>> 
> 
> 
> 
> -- 
> 
> A FreeBSD user

--
Bob Bishop
rb@gid.co.uk