From nobody Sun Jan 07 15:38:04 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T7Ltp0qtLz57FRW; Sun, 7 Jan 2024 15:38:10 +0000 (UTC) (envelope-from SRS0=GM30=IR=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4T7Ltn50P6z41wg; Sun, 7 Jan 2024 15:38:09 +0000 (UTC) (envelope-from SRS0=GM30=IR=quip.cz=000.fbsd@elsa.codelab.cz) Authentication-Results: mx1.freebsd.org; none Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id C20E7D788C; Sun, 7 Jan 2024 16:38:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quip.cz; s=private; t=1704641886; bh=957MXApsj9xlddtQHv/VyW8E2u9tQcdfWlSGlI4FFSk=; h=Date:Subject:To:References:From:In-Reply-To; b=Bjfvc/xkdYJzwV/5aZgJS/kvstrmKPvGTOdKxFWU6PvacCJu6lj+FXW45eut3GdRL aGIo0z1CvA+JmQQRdnGLFQML1eBr6YqkVoMNJZIMNMC/XWjriYFGb3uJzMVWWn+igP O8otu3zufJ2i03eyySuMrBO+SJKoVKUXRYxySsK4= Received: from [192.168.145.49] (ip-89-177-27-225.bb.vodafone.cz [89.177.27.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 29193D7888; Sun, 7 Jan 2024 16:38:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quip.cz; s=private; t=1704641885; bh=957MXApsj9xlddtQHv/VyW8E2u9tQcdfWlSGlI4FFSk=; h=Date:Subject:To:References:From:In-Reply-To; b=KcDUPi1Ljaay3dlxi4Pvm/iSpwFVkyD8VgIfarwyCj3DrpmY2hYy/5fZS6qMilNCd HD/bIgGz7lUwHB1Hv/3g7k7fWmXbWomp33EDPrnSHNAcLRxEPi2etgup9eg0VDQ0PY teyRDtv1m3BC3ATXj+JQlClOwFRACQ4no3svSrec= Message-ID: Date: Sun, 7 Jan 2024 16:38:04 +0100 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: FreeBSD 13.2-STABLE can not boot from damaged mirror AND pool stuck in "resilver" state even without new devices. Content-Language: en-US To: lev@FreeBSD.org, freebsd-fs , freebsd-stable , Alexander Motin References: From: Miroslav Lachman <000.fbsd@quip.cz> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4T7Ltn50P6z41wg X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ] On 07/01/2024 15:49, Lev Serebryakov wrote: > On 05.01.2024 18:28, Lev Serebryakov wrote: > >>     After that my server fails to boot, gtpzfsboot from second disk >> (ada1) reports several "zio_read error: 5" and >> >> ZFS: i/o error - all block copies unavailable >> ZFS: can't read MOS of pool zroot >> >>     after that. >  I've re-created pool from scratch > >  zpool create znewroot ada0p3 && zfs send zroot | zfs receive znewroot > && zpool destroy zroot && zpool attach znewroot ada0p3 ada1p3 > >  but gptzfsboot still can not boot from it with same diagnostics :-( How large are your disks in a question? I was bitten by this not a long time ago when migrating my 2TB pool by zfs send to larger disks (4TB), then I see the error: ZFS: i/o error - all block copies unavailable ZFS: can't read MOS of pool zroot As far as I search the internet it is caused by the boot code (later stage which is in a file in /boot directory) was moved too far from the beginning of the disk and some old BIOS cannot allow the system to continue booting. I am not a boot expert so my words can be wrong but I hope you get the point. It can be result of the system update, or zfs send | zfs recv. In my case the pool was unbootable in HP Miniserver Ge 8 but boots perfectly fine in an old Supermicro with X9SCA-F board. The problem is not in a pool, nor disks, nor FreeBSD but in a BIOS. I solved it by creating new mirrored pool of the size about 40GB at the beginning of the disks (40GB GPT partition for freebsd-zfs) where I installed the FreeBSD system and next freebsd-zfs partition covering the rest of the 4TB disks for data storage. Everything works fine. You can also have just a small /boot partition for the boot and later overlayed by main ZFS pool, but it seems to me as bad for maintaining. example of my partitions layout # gpart show -p => 40 7814037088 ada0 GPT (3.6T) 40 1024 ada0p1 freebsd-boot (512K) 1064 40960 ada0p2 efi (20M) 42024 83886080 ada0p3 freebsd-zfs (40G) 83928104 20971520 ada0p4 freebsd-swap (10G) 104899624 7707033600 ada0p5 freebsd-zfs (3.6T) 7811933224 2103904 - free - (1.0G) It can also be avoided if your machine supports EFI boot, but my HP Microserver Gen 8 does not support it. Kind regards Miroslav Lachman