Re: ZFS: i/o error - all block copies unavailable

On 1/6/2023 4:06 PM, John Doherty wrote:
> BIOS settings look fine. Boot option #1 is a Micron SSD as it should 
> be. It's not clear which of the two installed devices like that it is. 
> There's one on SATA port 0 and an identical one on SATA port 1.
>
> Also, occurred to me after my previous message that the machine can't 
> be trying to boot from zp0, there's no OS installed there. If I let it 
> try to boot normally, it gets far enough that I can ping its network 
> interfaces, so it must be trying to boot from the zroot pool.
>
> Now wondering whether there's a problem with one of the two SSDs 
> configured as the zroot pool. So far, don't see how to choose whether 
> it boots from SATA P0 or SATA P1 but there's got to be a way, I think.
>
Does it mount anything from zp0 like /usr thats required for the OS to 
fully boot ?  Can you not use ipmi to bring up the console and log in 
via single user mode ?  You can change the boot pref in the BIOS (on 
most motherboards anyways) but if its a mirror, beyond the boot loader I 
dont think it should make much of a difference or even hit F11 to bring 
the boot menu at bios time. But its when you move to RELENG_13 with the 
different zfs features you need to be really careful to update the boot 
loaders in order to recognize the new features of the pool.

     ---Mike


> On Fri 2023-01-06 12:17 PM MST -0700, <bsdlists@jld3.net> wrote:
>
>> Thanks for your response, that's a good idea. The HDDs are connected 
>> to a Supermicro AOC-S3008L-L8e, which has a Broadcom 3008 SAS 
>> controller supported by mpt(4). I have to take a break now but will 
>> try getting into the BIOS next.
>>
>> On Fri 2023-01-06 12:13 PM MST -0700, <mike@sentex.net> wrote:
>>
>>
>>> On 1/6/2023 2:08 PM, John Doherty wrote:
>>>
>>
>>>> Posting this to freebsd-fs because the problem appears to be 
>>>> related to ZFS.
>>>>
>>>> I have a system running FreeBSD 12.3-RELEASE to which I have only 
>>>> remote access (about 3,300 miles from where I am).
>>>>
>>>> This morning I ran "freebsd-update fetch" and "freebsd-update 
>>>> install" on it. This was to get it up to date with 12.3 in 
>>>> preparation for a further update to 13.1-RELEASE.
>>>>
>>>> I rebooted the machine after "freebsd-update install" and it 
>>>> doesn't boot all the way. I connected to its IPMI and by recording 
>>>> the screen while trying to boot it, I was able to capture error 
>>>> messages that went by too fast to read otherwise:
>>>>
>>>> ZFS: i/o error - all block copies unavailable
>>>> ZFS: can't read MOS of pool zp0
>>>>
>>
>>>
>>> Is the disk controller's driver being loaded by the kernel ? What is 
>>> the driver ?  Is it possible that the disk order has been messed up 
>>> and its trying to boot not from the boot pool, but from disks in zp0 
>>> and not trying zroot ? If you have ipmi, try and go into the BIOS 
>>> and make sure the disk its trying to boot from first is indeed part 
>>> of zroot
>