Re: FreeBSD 13.2-STABLE can not boot from damaged mirror AND pool stuck in "resilver" state even without new devices.

From: Lev Serebryakov <lev_at_FreeBSD.org>
Date: Sun, 07 Jan 2024 15:04:36 UTC
On 07.01.2024 15:51, Karl Denninger wrote:

>>>     After that my server fails to boot, gtpzfsboot from second disk (ada1) reports several "zio_read error: 5" and
>>>
>>> ZFS: i/o error - all block copies unavailable
>>> ZFS: can't read MOS of pool zroot
>>>
>>>     after that.
>>  I've re-created pool from scratch
>>
>>  zpool create znewroot ada0p3 && zfs send zroot | zfs receive znewroot && zpool destroy zroot && zpool attach znewroot ada0p3 ada1p3
>>
>>  but gptzfsboot still can not boot from it with same diagnostics :-(
>>
> Are you SURE gptzfsboot is up to date on that second drive?
> 
> If its older that can definitely happen.
  I'm 100% sure that gtpzfsboot are from 13.2-RELEASE ISO (system is 13.2-STABLE stable/13-n257023-d32f39bd5fbf) on both disks. I've reinstalled it more than 10 times now...

  Now BIOS set to boot from ada0.

  Now I'm booting via "gptboot" at ada0p1/ada1p1 and small freebsd-ufs partition on ada0p2/ada0p3 with only "/boot" as temporary measure (this partition was created instead of swap temporary, to revive server).

  Unfortunately, it is hard to experiment with different bootcode, as it needs remote console which is in limited availability (Hetzner!) and don't give you access to BIOS anyhow.

  (it is pity, that our new bootloaders have less features than old 512-byte "boot0" which allows to select different HDDs without any BIOS interaction! We can several megabytes of code on these new bootloaders, and still they are DUMB)

-- 
// Lev Serebryakov