Re: FreeBSD 13.2-STABLE can not boot from damaged mirror AND pool stuck in "resilver" state even without new devices.

On 01/08/2024 5:15 am, Lev Serebryakov wrote:
> On 08.01.2024 1:15, Larry Rosenman wrote:
> 
>>>> So, if this is a mirror, then ada0 blank and ada1 with good data, in 
>>>> theory
>>>> you should be fine. However, perhaps ZFS is finding that there's an 
>>>> error from
>>>> ada1 for real. Does all of ada1 read with a simple dd?
>>>   Yep, it is read with dd, I've checked it
>>> 
>>>> Not sure about the losing devices you described later on.
>>>> 
>>>>      > ZFS: i/o error - all block copies unavailable
>>>>      > ZFS: can't read MOS of pool zroot
>>>>      >
>>>>      >
>>>>      >   To be honest, I thinks there is something else. Because 
>>>> sequence of events were (sorry, too long, but I think, tht every 
>>>> detail matters here):
>>>> 
>>>> 
>>>> Yea. There's something that's failing, which zio_read is woefully 
>>>> under reporting for our diagnostic efforts. And/or something is
>>>> getting confused by the blank disk and/or the partially resilvered 
>>>> disk.
>>> 
>>>   My theory, that something is confused when one disk is 512/4096 and 
>>> other is 512/512.
>>> 
>>>   I want to check it on VM, but can not find VM that both (1) allows 
>>> CMS boot and (2) allows to configure logical and physical sector of 
>>> virtual HDD.
>>> 
>>>   bhyve could configure sector sizes, but doesn't support BIOS, and 
>>> VBox and qemu-system can not emulate sector sizes (or I can not 
>>> google proper configuration).
>> 
>> When I first saw this, I wonder what ashift is set to on the pool?
>   old pool was with ashift=9, but new one is with ashift=12.
I wonder if the ashif=9 caused the issue when you added the 4Kn disk?
-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 214-642-9640                 E-Mail: ler@lerctr.org
US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106