From nobody Tue Feb 04 10:18:06 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YnK7j2sjVz5mXYP for ; Tue, 04 Feb 2025 10:18:09 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YnK7j2KJlz3DpT for ; Tue, 04 Feb 2025 10:18:09 +0000 (UTC) (envelope-from avg@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1738664289; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cUidbaljyC5nFVjqI47BrSNasTU3sCCSVarp417bTRQ=; b=rFIfkZlZEe2STQXiEdcpojX56k6EmG8nhaE4kyvQPJukwYwqsxoBF61LZac9roozEvIMXv JH+2hNmRuTJc24Y/UlMd/wtJcz96dJpJSsmVVqkv+/sqI0HUOFKwAMP/e68uAEClIxHyas bzwMMWPItBh/eRH5KHx/mCMRczPnyhhSb89QnMDcbGUcizyIJFbejVQt65XvRDZgSVVKwW dL7yZlZYiX/m62REev/kXxr00DVCyFvhMi2h9F2efX8tqa+H9Erh0ubkC189S2xtxzPQDG FRgZc51GEqlMCGHt0A1/Smjq7ZQAV0RyKGVzaibWbB8KpMeXO21rB6gkuamMtQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1738664289; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cUidbaljyC5nFVjqI47BrSNasTU3sCCSVarp417bTRQ=; b=Lh4edWWPHHlxPAqINMdDGFj8njfWSSTWBb2UF8DshvljDiifhwexCQFjceMksqWfzVhed6 PTVMi8E+B7XMi88SA4jSy7cApDNzmvf8qR8fGSbYEl9FszMU2kXXgIkMIThDjB5Gxiql28 tve16cHqcZ8QjzCDKr3/TLzMuseU/1RXTZkda2lLNd/3L4uudjWVg+SvBVHHOmGSH+xl0e D2QzBT2nUNvh71cjoV2KLIeTRbNnriWMSuPTYzCgrMQOmEmfb3/+SbBX/OO2xyuMW19LIJ Fdti3hr3WmJnMT2gFPGP1SUeRK92RQpjPmzfc1Ooc9pxpmVpeFOSYBpEKZL5pg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1738664289; a=rsa-sha256; cv=none; b=i9ogLNSIuYgH3bLvf3DAT5dhZJnxY94JYQ7ioIpS4lu4+LDZ9TKJ1OA/fI1Cjwx/2yKJFh GXtQ0Eyd+ob9SidOr/fmrlWK+sDu6pdTYimuhCo0RLf5mpI+PiMSNTwIDdEPUDY1fd3asN zkJIgusKcbEwL2/pyIVQKNcwy2FwECvG+YiYs4SHKqJyQhLG96lMHLlt0bUJFg/lcv5E9s lFg8EeAREyr5KF8gEQqlyAfHeyoSBkoHQd/PGwFHAaP4JN3aPMV1WuYmKbpXvWNWUTe2Ox 1nV2FR8tU4dNGrezM19/tpGGphapbOg/3Nu1IXQZlQ/e5zyAFieQ4tZUz2mg3w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from [192.168.0.88] (unknown [93.188.39.137]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: avg/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4YnK7h6lLwzBkN for ; Tue, 04 Feb 2025 10:18:08 +0000 (UTC) (envelope-from avg@FreeBSD.org) Message-ID: Date: Tue, 4 Feb 2025 12:18:06 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: ZFS: Rescue FAULTED Pool To: freebsd-current@freebsd.org References: <20250129112701.0c4a3236@freyja> <20250130123354.2d767c7c@thor.sb211.local> <980401eb-f8f6-44c7-8ee1-5ff0c9e1c35c@freebsd.org> <20250201095656.1bdfbe5f@thor.sb211.local> Content-Language: en-US From: Andriy Gapon In-Reply-To: <20250201095656.1bdfbe5f@thor.sb211.local> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 01/02/2025 10:57, A FreeBSD User wrote: > Hello, this exactly happens when trying to import the pool. Prior to the loss, device da1p1 > has been faulted with numbers in the colum/columns "corrupted data"/further not seen now. > > > ~# zpool import > pool: BUNKER00 > id: XXXXXXXXXXXXXXXXXXXX > state: FAULTED > status: The pool metadata is corrupted. > action: The pool cannot be imported due to damaged devices or data. > The pool may be active on another system, but can be imported using > the '-f' flag. > see:https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-72 > config: > > BUNKER00 FAULTED corrupted data > raidz1-0 ONLINE > da2p1 ONLINE > da3p1 ONLINE > da4p1 ONLINE > da7p1 ONLINE > da6p1 ONLINE > da1p1 ONLINE > da5p1 ONLINE > > > ~# zpool import -f BUNKER00 > cannot import 'BUNKER00': I/O error > Destroy and re-create the pool from > a backup source. > > > ~# zpool import -F BUNKER00 > cannot import 'BUNKER00': one or more devices is currently unavailable Too late now, but another useful command for situations like this is zdb -G BUNKER00 It would print a log of various pool import actions. E.g., on a good pool: # zdb -G rpool ZFS_DBGMSG(zdb) START: spa.c:5694:spa_open_common(): spa_open_common: opening rpool spa_misc.c:419:spa_load_note(): spa_load(rpool, config trusted): LOADING vdev.c:162:vdev_dbgmsg(): disk vdev '/dev/gpt/S6PEN.rpool': best uberblock found for spa rpool. txg 61892397 spa_misc.c:419:spa_load_note(): spa_load(rpool, config untrusted): using uberblock with txg=61892397 spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading checkpoint txg spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading indirect vdev metadata spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Checking feature flags spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading special MOS directories spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading properties spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading AUX vdevs spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading vdev metadata spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading dedup tables spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Loading BRT spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Verifying Log Devices spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Verifying pool data spa_misc.c:419:spa_load_note(): spa_load(rpool, config trusted): spa_load_verify found 0 metadata errors and 4 data errors spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Calculating deflated space spa_misc.c:2311:spa_import_progress_set_notes_impl(): 'rpool' Starting import spa.c:8925:spa_async_request(): spa=rpool async request task=2048 spa_misc.c:419:spa_load_note(): spa_load(rpool, config trusted): LOADED ZFS_DBGMSG(zdb) END On a bad pool, the log may have helped to identify the exact problem. -- Andriy Gapon