From nobody Sun Jan 07 21:19:33 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T7VSm0qB4z561Gl; Sun, 7 Jan 2024 21:19:36 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T7VSl6ttKz46hp; Sun, 7 Jan 2024 21:19:35 +0000 (UTC) (envelope-from lev@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704662376; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Tg/IoX2r7626nl2FGGM3y4zlJB3O1AF5MWoK1EcwkpU=; b=k/jEy2sDVd1RCMH84prpz8uR7fa9/dZ3R7cixAVMOL51ksrrOexHYw5Uk8cYyLvlnyR5aP Gjeeqi7/YhPapzj/y+A2JAE7FiYc6Q6cg8OJMWBuInKkd7QfpAG4dgtBcdEFkoFD7T6wMz OSakZFlumu6821IUObA61/TeeImRmAGVOrfTqlpLZFOb67bpHC3GZsuhIkmw+5q/ScZuJA TDRHvz2cOfQnxUxE2pMcJuTHOE48VgLFGc0I1hQpwv21AWaaUi9/xSBovkoGqR3n3YWiHt EBdxuKOhRqO8oM5AWl1ydLQIKB2cW86Y+5xuwNYV+e/mIzh/UPmaoHo47WTcuw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704662376; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Tg/IoX2r7626nl2FGGM3y4zlJB3O1AF5MWoK1EcwkpU=; b=bFDOLEhKyIU5N/of/+7Tb4pNfVkXWyTabzh6LucyFXnYEc/6SeRIge4jfW68Q/+Jg+Dzru eiI2wslKGzKH3BtntFK+oakNZjDHFV/iNBvpagksDiXtX2Xxwm/3+NeAGb1u3xWa6aSrxB nQZlH1j36QdYtlIM+xVTbWD/O5RtFXigj0Xz66+VGUZOHFXMOq//S/L66o6VNLq1ZkzW7u p1GfJgIfCnma65nQZwlY8Ud97V+Qze7YiJ8u3v33Xp/nF/FTXiIuBJ0iyyEwTFFGwpX1eY 6FwNuUyC8UrJ8eVEdTYB4uR+1tq75avKQz4hv8gLCGn34jgRMXAW3xmynzsrGw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1704662376; a=rsa-sha256; cv=none; b=T+pn7GkMuahbcvW2yRNY3NUlAjJrDo7sutAWo16ezTyy9jZ7dTAoUBp491IUig9BpxoiKV 6U5d58u4IfQnAUQSpVHYWfhs5yh0w9SYkW0oHDe3NMk2CyThAAcZ/RGS3/oEW/UxQMy6kt KCWD+Pb1L0lhVIvvGwgmTBU4AwCn7Bd6SkcZSwYyNn+xYScVfwpAsY6Q8Mv90bWGuTL/Kg EuwlZ3TIeR8qigbHO1cvDgffuhzqTGkkQdXdUFRVevb8wOMgG9aVcoclf+fuESJ89HnC+E pjwnKKBLPe08XaSqdmvhN/iCSU9nYeBnjr8fT4DTcZ+5/rpFQGxUa+CnrHbpPQ== Received: from onlyone.not-for.work (onlyone.not-for.work [IPv6:2a01:4f8:201:6350::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: lev/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4T7VSl5FyfzZfS; Sun, 7 Jan 2024 21:19:35 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from [192.168.136.24] (83-84-181-95.cable.dynamic.v4.ziggo.nl [83.84.181.95]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id EC91F299; Mon, 8 Jan 2024 00:19:33 +0300 (MSK) Message-ID: <962b242d-546f-46ce-9eb2-9bd2a10f4608@FreeBSD.org> Date: Sun, 7 Jan 2024 22:19:33 +0100 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Reply-To: lev@FreeBSD.org Subject: Re: FreeBSD 13.2-STABLE can not boot from damaged mirror AND pool stuck in "resilver" state even without new devices. Content-Language: en-US To: Warner Losh Cc: freebsd-fs , freebsd-stable References: <065f4f5c-f38b-45f4-b7e7-5248f871f7e6@FreeBSD.org> <2f91eeb7-430b-49e2-817b-5acd0f445fe9@FreeBSD.org> From: Lev Serebryakov Organization: FreeBSD In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 07.01.2024 22:15, Warner Losh wrote: > So, if this is a mirror, then ada0 blank and ada1 with good data, in theory > you should be fine. However, perhaps ZFS is finding that there's an error from > ada1 for real. Does all of ada1 read with a simple dd? Yep, it is read with dd, I've checked it > Not sure about the losing devices you described later on. > > > ZFS: i/o error - all block copies unavailable > > ZFS: can't read MOS of pool zroot > > > > > >   To be honest, I thinks there is something else. Because sequence of events were (sorry, too long, but I think, tht every detail matters here): > > > Yea. There's something that's failing, which zio_read is woefully under reporting for our diagnostic efforts. And/or something is > getting confused by the blank disk and/or the partially resilvered disk. My theory, that something is confused when one disk is 512/4096 and other is 512/512. I want to check it on VM, but can not find VM that both (1) allows CMS boot and (2) allows to configure logical and physical sector of virtual HDD. bhyve could configure sector sizes, but doesn't support BIOS, and VBox and qemu-system can not emulate sector sizes (or I can not google proper configuration). -- // Lev Serebryakov