From nobody Fri Jan 06 21:15:43 2023 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NpbjD41lvz2qkj5 for ; Fri, 6 Jan 2023 21:15:44 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [IPv6:2607:f3e0:0:1::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smarthost1.sentex.ca", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NpbjD1Zx6z3vRB for ; Fri, 6 Jan 2023 21:15:44 +0000 (UTC) (envelope-from mike@sentex.net) Authentication-Results: mx1.freebsd.org; none Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [199.212.134.19]) by smarthost1.sentex.ca (8.16.1/8.16.1) with ESMTPS id 306LFhtY064919 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=FAIL); Fri, 6 Jan 2023 16:15:43 -0500 (EST) (envelope-from mike@sentex.net) Received: from [IPV6:2607:f3e0:0:4:9423:c96b:ab98:2f26] ([IPv6:2607:f3e0:0:4:9423:c96b:ab98:2f26]) by pyroxene2a.sentex.ca (8.16.1/8.15.2) with ESMTPS id 306LFgJH084361 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Fri, 6 Jan 2023 16:15:43 -0500 (EST) (envelope-from mike@sentex.net) Message-ID: <8d73d0b1-a175-79b7-f74d-f19977dca9b1@sentex.net> Date: Fri, 6 Jan 2023 16:15:43 -0500 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.6.1 Subject: Re: ZFS: i/o error - all block copies unavailable Content-Language: en-US To: John Doherty , freebsd-fs References: <9DDC68E1-4321-45D9-831E-A73F70C66191@jld3.net> <48b9848c-2e47-0d4b-f8b0-6a8ef7a7c982@sentex.net> <867AADBE-60F8-4B54-BBE8-875D381DA296@jld3.net> <3C4D1747-53D4-4A5A-9153-5777D2D895CE@jld3.net> From: mike tancsa In-Reply-To: <3C4D1747-53D4-4A5A-9153-5777D2D895CE@jld3.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.84 X-Rspamd-Queue-Id: 4NpbjD1Zx6z3vRB X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On 1/6/2023 4:06 PM, John Doherty wrote: > BIOS settings look fine. Boot option #1 is a Micron SSD as it should > be. It's not clear which of the two installed devices like that it is. > There's one on SATA port 0 and an identical one on SATA port 1. > > Also, occurred to me after my previous message that the machine can't > be trying to boot from zp0, there's no OS installed there. If I let it > try to boot normally, it gets far enough that I can ping its network > interfaces, so it must be trying to boot from the zroot pool. > > Now wondering whether there's a problem with one of the two SSDs > configured as the zroot pool. So far, don't see how to choose whether > it boots from SATA P0 or SATA P1 but there's got to be a way, I think. > Does it mount anything from zp0 like /usr thats required for the OS to fully boot ?  Can you not use ipmi to bring up the console and log in via single user mode ?  You can change the boot pref in the BIOS (on most motherboards anyways) but if its a mirror, beyond the boot loader I dont think it should make much of a difference or even hit F11 to bring the boot menu at bios time. But its when you move to RELENG_13 with the different zfs features you need to be really careful to update the boot loaders in order to recognize the new features of the pool.     ---Mike > On Fri 2023-01-06 12:17 PM MST -0700, wrote: > >> Thanks for your response, that's a good idea. The HDDs are connected >> to a Supermicro AOC-S3008L-L8e, which has a Broadcom 3008 SAS >> controller supported by mpt(4). I have to take a break now but will >> try getting into the BIOS next. >> >> On Fri 2023-01-06 12:13 PM MST -0700, wrote: >> >> >>> On 1/6/2023 2:08 PM, John Doherty wrote: >>> >> >>>> Posting this to freebsd-fs because the problem appears to be >>>> related to ZFS. >>>> >>>> I have a system running FreeBSD 12.3-RELEASE to which I have only >>>> remote access (about 3,300 miles from where I am). >>>> >>>> This morning I ran "freebsd-update fetch" and "freebsd-update >>>> install" on it. This was to get it up to date with 12.3 in >>>> preparation for a further update to 13.1-RELEASE. >>>> >>>> I rebooted the machine after "freebsd-update install" and it >>>> doesn't boot all the way. I connected to its IPMI and by recording >>>> the screen while trying to boot it, I was able to capture error >>>> messages that went by too fast to read otherwise: >>>> >>>> ZFS: i/o error - all block copies unavailable >>>> ZFS: can't read MOS of pool zp0 >>>> >> >>> >>> Is the disk controller's driver being loaded by the kernel ? What is >>> the driver ?  Is it possible that the disk order has been messed up >>> and its trying to boot not from the boot pool, but from disks in zp0 >>> and not trying zroot ? If you have ipmi, try and go into the BIOS >>> and make sure the disk its trying to boot from first is indeed part >>> of zroot >