From nobody Fri Feb 07 13:30:46 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YqFH220FVz5nPF9 for ; Fri, 07 Feb 2025 13:31:10 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from gid2.gid.co.uk (ns0.gid.co.uk [IPv6:2001:470:94de::240]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gid2.gid.co.uk", Issuer "gid2.gid.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YqFH11jfgz3qKN; Fri, 07 Feb 2025 13:31:09 +0000 (UTC) (envelope-from rb@gid.co.uk) Authentication-Results: mx1.freebsd.org; none Received: from mx0.gid.co.uk (mx0.gid.co.uk [194.32.164.250]) by gid2.gid.co.uk (8.15.2/8.15.2) with ESMTP id 517DV2jE086001; Fri, 7 Feb 2025 13:31:02 GMT (envelope-from rb@gid.co.uk) Received: from smtpclient.apple ([194.32.164.16]) by mx0.gid.co.uk (8.14.2/8.14.2) with ESMTP id 517DUuC1065808; Fri, 7 Feb 2025 13:30:57 GMT (envelope-from rb@gid.co.uk) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: ZFS: Rescue FAULTED Pool From: Bob Bishop In-Reply-To: <20250207094848.3fcdd20a@thor.sb211.local> Date: Fri, 7 Feb 2025 13:30:46 +0000 Cc: "freebsd-current@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <223C2C8D-9F54-448E-9107-A1AEA5659061@gid.co.uk> References: <20250129112701.0c4a3236@freyja> <20250130123354.2d767c7c@thor.sb211.local> <980401eb-f8f6-44c7-8ee1-5ff0c9e1c35c@freebsd.org> <20250201095656.1bdfbe5f@thor.sb211.local> <20250207094848.3fcdd20a@thor.sb211.local> To: A FreeBSD User , Andriy Gapon X-Mailer: Apple Mail (2.3776.700.51.11.1) X-Rspamd-Queue-Id: 4YqFH11jfgz3qKN X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] Hi, > On 7 Feb 2025, at 08:48, A FreeBSD User = wrote: >=20 > Am Tue, 4 Feb 2025 12:18:06 +0200 > Andriy Gapon schrieb: >=20 >> On 01/02/2025 10:57, A FreeBSD User wrote: >>> Hello, this exactly happens when trying to import the pool. Prior to = the loss, device da1p1 >>> has been faulted with numbers in the colum/columns "corrupted = data"/further not seen now. >>>=20 >>>=20 >>> ~# zpool import >>> pool: BUNKER00 >>> id: XXXXXXXXXXXXXXXXXXXX >>> state: FAULTED >>> status: The pool metadata is corrupted. >>> action: The pool cannot be imported due to damaged devices or data. >>> The pool may be active on another system, but can be = imported using >>> the '-f' flag. >>> see:https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-72 >>> config: >>>=20 >>> BUNKER00 FAULTED corrupted data >>> raidz1-0 ONLINE >>> da2p1 ONLINE >>> da3p1 ONLINE >>> da4p1 ONLINE >>> da7p1 ONLINE >>> da6p1 ONLINE >>> da1p1 ONLINE >>> da5p1 ONLINE >>>=20 >>>=20 >>> ~# zpool import -f BUNKER00 >>> cannot import 'BUNKER00': I/O error >>> Destroy and re-create the pool from >>> a backup source. >>>=20 >>>=20 >>> ~# zpool import -F BUNKER00 >>> cannot import 'BUNKER00': one or more devices is currently = unavailable =20 >>=20 >>=20 >> Too late now, but another useful command for situations like this is >> zdb -G BUNKER00 >> It would print a log of various pool import actions. >=20 > Not "too late", very much appreciated, such helpful = worst-case-scenario would have a nice > place in a small section "desaster recovery" in the handbook, wouldn't = it? The handbook is > quite superficial at that point ... +1 > Kind regards, > Oliver >>=20 >> E.g., on a good pool: >> # zdb -G rpool >>=20 >> ZFS_DBGMSG(zdb) START: >> spa.c:5694:spa_open_common(): spa_open_common: opening rpool >> spa_misc.c:419:spa_load_note(): spa_load(rpool, config trusted): = LOADING >> vdev.c:162:vdev_dbgmsg(): disk vdev '/dev/gpt/S6PEN.rpool': best = uberblock found=20 >> for spa rpool. txg 61892397 >> spa_misc.c:419:spa_load_note(): spa_load(rpool, config untrusted): = using=20 >> uberblock with txg=3D61892397 >> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading = checkpoint txg >> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading = indirect=20 >> vdev metadata >> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' = Checking feature flags >> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading = special=20 >> MOS directories >> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading = properties >> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading = AUX vdevs >> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading = vdev metadata >> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading = dedup tables >> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading = BRT >> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' = Verifying Log Devices >> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' = Verifying pool data >> spa_misc.c:419:spa_load_note(): spa_load(rpool, config trusted): = spa_load_verify=20 >> found 0 metadata errors and 4 data errors >> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' = Calculating=20 >> deflated space >> spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' = Starting import >> spa.c:8925:spa_async_request(): spa=3Drpool async request task=3D2048 >> spa_misc.c:419:spa_load_note(): spa_load(rpool, config trusted): = LOADED >> ZFS_DBGMSG(zdb) END >>=20 >> On a bad pool, the log may have helped to identify the exact problem. >>=20 >=20 >=20 >=20 > --=20 >=20 > A FreeBSD user -- Bob Bishop rb@gid.co.uk