From nobody Sun Jan 07 15:04:36 2024 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T7L876sWmz57BNf for ; Sun, 7 Jan 2024 15:04:39 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T7L876Hjwz4tj6; Sun, 7 Jan 2024 15:04:39 +0000 (UTC) (envelope-from lev@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704639879; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pr+GvXPfwT1f5YnvdMGwMSjSB/QoWjWxRi7spPr6M7c=; b=sD6fmblpy9JlvPdLXuY0COsk00hnelleGnjW3tyOEoimjCIAPWGmB3NC+TlrmTFPYg9W7g TaQiHeu6t/LJylF0re+91XA5GnnQXoxX+1qBaiYuEPwenmopJxIyjrkmKkwPw6M32ELB15 kzrnMUIQQimlsfbFJmh0sMwT3SvL7OxEpz74f5wWgS9u9zhraxdrDYx+ZfWg1qvHu7gcFr i7YhL/MleeFAYLZwU+4gayDGd/HSpHenrpRsoY8kohqBDnutXlHYaMH69VQyBFQxw0X4Dj LmPC98z5KJlO11BJV0FJBCGjYoulYbE0RSVJkKNIJpJJir4bWhREyKkZzZ3uZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704639879; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pr+GvXPfwT1f5YnvdMGwMSjSB/QoWjWxRi7spPr6M7c=; b=ASz7aPT4PdWQ9YE5B72N9KodQ0ZT/hDo00v4aAJXM2Es8ptEthNEcQJOO1c56tO7vVpI1C IVNA9hLpfFSxMmSWkbs8RhRzFhShiLIkpQCWCVo+q3rKF/I77aCp0VFAIrnjbbYerWBvgX bITVYXCgQ3RsVRvJagCajAnpFM3ydxJaUulO9mOseGoZ8FxfoeEqTnpVJiRPmfBZ65aRY9 JmkvlobJKdtVQ73mpKxqxd9Xvwd0zskqXlwWlmz1f6CvzqzYsxXHWbbXbHd39HUalbhj33 zifB8bRW7GIY/oAPz/iZJCF90cxOF3+GpHCDTiOLxs3EBUVWSPbi3xaocRLoRA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1704639879; a=rsa-sha256; cv=none; b=jdQUjAslKTEhkQtwkoNfC3I1NhCuXvQhKpDzZMhyrywJuNw4igKcE4QnmHU3JTl+Y0svB/ uAHhpzq/FDvbZi5q4OBKVnj8RIALc2COJh16tlJ4LHuU0GZczIM+zccW03XuEsjYuK75fG 5aSmIWGabJQ73TFEU+yNWkpLgVmK39e//TgNJ+dgViEC8xWC+iDATisCoU9+QsZMutNfaR 77H72GBbKv1Ci8rb/AwUwYTabAqV+XvxPzx7iDgXbbOU3zrOwrr6oorhiYap1TwQFCyFcD 84hwKeon7h/YVbigT2qPUXd4k7Yy5wuhnG6SBfKWoO9usjW4482Nkx1cK00Czw== Received: from onlyone.not-for.work (onlyone.not-for.work [148.251.9.81]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: lev/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4T7L874TsWzSRk; Sun, 7 Jan 2024 15:04:39 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from [IPV6:2001:470:923f:1:795b:6f6e:6bbc:d02c] (unknown [IPv6:2001:470:923f:1:795b:6f6e:6bbc:d02c]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id 5D8EC294; Sun, 7 Jan 2024 18:04:37 +0300 (MSK) Message-ID: <68958573-e2cf-4af2-9f32-0fb25809169a@FreeBSD.org> Date: Sun, 7 Jan 2024 16:04:36 +0100 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: FreeBSD 13.2-STABLE can not boot from damaged mirror AND pool stuck in "resilver" state even without new devices. Content-Language: en-US To: Karl Denninger , stable@freebsd.org References: <11878eab-a60e-4d5c-8eef-b9a87c73dfc5@denninger.net> From: Lev Serebryakov Reply-To: lev@FreeBSD.org Organization: FreeBSD In-Reply-To: <11878eab-a60e-4d5c-8eef-b9a87c73dfc5@denninger.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 07.01.2024 15:51, Karl Denninger wrote: >>>     After that my server fails to boot, gtpzfsboot from second disk (ada1) reports several "zio_read error: 5" and >>> >>> ZFS: i/o error - all block copies unavailable >>> ZFS: can't read MOS of pool zroot >>> >>>     after that. >>  I've re-created pool from scratch >> >>  zpool create znewroot ada0p3 && zfs send zroot | zfs receive znewroot && zpool destroy zroot && zpool attach znewroot ada0p3 ada1p3 >> >>  but gptzfsboot still can not boot from it with same diagnostics :-( >> > Are you SURE gptzfsboot is up to date on that second drive? > > If its older that can definitely happen. I'm 100% sure that gtpzfsboot are from 13.2-RELEASE ISO (system is 13.2-STABLE stable/13-n257023-d32f39bd5fbf) on both disks. I've reinstalled it more than 10 times now... Now BIOS set to boot from ada0. Now I'm booting via "gptboot" at ada0p1/ada1p1 and small freebsd-ufs partition on ada0p2/ada0p3 with only "/boot" as temporary measure (this partition was created instead of swap temporary, to revive server). Unfortunately, it is hard to experiment with different bootcode, as it needs remote console which is in limited availability (Hetzner!) and don't give you access to BIOS anyhow. (it is pity, that our new bootloaders have less features than old 512-byte "boot0" which allows to select different HDDs without any BIOS interaction! We can several megabytes of code on these new bootloaders, and still they are DUMB) -- // Lev Serebryakov