From nobody Mon Sep 16 23:21:35 2024 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4X71Bm3lBhz5WP5C for ; Mon, 16 Sep 2024 23:21:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4X71Bl6hrlz4YYR for ; Mon, 16 Sep 2024 23:21:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1726528895; a=rsa-sha256; cv=none; b=x/z/OWHspW2hn78CWrBvPYbFgbTVr02Cyblm8AVsF5B2FGm+91MkBxPxWqil/Lt5y4dL3d bkvotvB9XPeyZypqU8Gbv4GN/iRzo1+wxjcJOkxB3byvBjE1UoxK82Ib3kRLJ+J+LMo6Xk /iuJzZMlARb7yZsDnxS/8ZXMJYxpOqySO79LQoyKX8rb8edu3AqbeGc/uWIoDhyTJkSVaJ ibf3hj/Z3LFTn+bMyYxgX1/ud72iDaZfoXqMaky05xACFFYUWGgG4BnYg0e4FGKCE2+JbP J/6pAMkhPo325xEJxXE6j3iiR2bW7qYFcoIA3XsQ7+ikkSQoAqt7huHAlvYRdQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1726528895; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=zw6S0EWiyb+LFbd4Epk+qCwJ9bDJw8v7kYy2P1qas44=; b=HYbXDAeIWCNHdAtmQHOPsR35rQCCuk1mXi/crgm7wyDYYFQq5LbWjUwY6ejuc79glIRSl8 esLqcyLNgsVITxmap7o2Qy5ZjB9+fhrRMWotqIKB+UOJGw7aKI8Y4J8+baO7vtEHFaMQ9D oCoEZivCp5SQddwIIJueGMVDDVXPGPc8epFrRlJOyZl6f39KAPTpyXCbWy5yuFwftchzkN C7/Wz2ZiQP6w3UMmgw7Hi2bIhflx1m+5Hoax4NTuSBW9UdiS8F/dN+fEOi6sHJJpP+d2f5 FSsW8XT3BlFK4MxeYQ9M9HyX5Xp3vOmPhzT7zyk3TS6CRCvB2ko5Xs4e/SKvxw== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4X71Bl4Bm8zFQV for ; Mon, 16 Sep 2024 23:21:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 48GNLZkR059561 for ; Mon, 16 Sep 2024 23:21:35 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 48GNLZwl059560 for bugs@FreeBSD.org; Mon, 16 Sep 2024 23:21:35 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 281547] nvd->nda + /dev/diskid + zfs triggers locking issues and partition not in /dev Date: Mon, 16 Sep 2024 23:21:35 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 14.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: russell.stuart@akips.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter attachments.created Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@FreeBSD.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D281547 Bug ID: 281547 Summary: nvd->nda + /dev/diskid + zfs triggers locking issues and partition not in /dev Product: Base System Version: 14.1-STABLE Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: russell.stuart@akips.com Created attachment 253614 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D253614&action= =3Dedit Problem 1 configuration just before reboot The issue is that after doing a FreeBSD 13 -> 14, the FreeBSD 14 system fai= ls to boot.=20=20=20 Two examples are given here that look different, but they share a lot of co= mmon factors and workarounds are the same, so I suspect the underlying cause is = the same. The workaround for both is adding either or both of these lines to /boot/loader.conf and ensure they are preserved during the 13 -> 14 upgrade: a. hw.nvme.use_nvd=3D"1" b. kern.geom.label.disk_ident.enable=3D"0" I suspect https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D241541 is the= same issue. Problem 1: After the upgrade from FreeBSD 13 -> 14, 14 fails to boot. The console displays problem-1-screenshot.png after the failure happens. It contains the ZFS WARING lines shown below repeated many times, whereupon it gives up and displays the "mountpoint>" prompt: ZFS WARNING: Unable to open diskid/DISK-2128309CF86Cp3 for writing (error=3D1) ZFS WARNING: Unable to open diskid/DISK-2128309CF86Cp4 for writing (error=3D1) ZFS WARNING: Unable to open gpt/akips-home for writing (error=3D1) Mounting from zfs:akips/ROOT/1 failed with error 1 Loader variables: vfs.root.mountfrom=3Dzfs:akips/ROOT/1 <....elided....> mountroot> I won't describe how here, but from this state I've rebooted back to FreeBS= D 13 and got the same error. Problem 2: After the upgrade from 13 -> 14, 14 fails to boot. The console displays problem-2-screenshot.png after the failure happens. The error mes= sage says it fails to boot because /dev/nda0p1 can't be mounted as /boot/efi. T= he reason if fails to mount is /dev/nvd0p1 doesn't exist. While /dev/nvd0 does exist (it's a symlink to nda0), no partition devices exist (ie, neither nda= 0p1 nor nvd0p1), so attaching the swap partition /dev/nvd0p2 also failed. All files under /dev after the reboot are shown in the attachment problem-2-lr-lR-dev.txt. Since this is a temporary boot environment, reboo= ting reverts back to the FreeBSD 13 root partition that worked moments ago. It = now fails with the same error. Reproduction ------------ Some bits are common to both: Hardware: - amd64. - A single nvme drive, at least 50GB. - No other drives connected. - 8GB RAM. Software: - Very specific disk layout (see the problem descriptions below). - Doing an upgrade from FreeBSD 13 (nvd) to FreeBSD 14 (nda). I can provide USB and .iso boot images that reproduce the problem if you ha= ve the hardware described above. I reproduced it on a NUC and VMWare. I susp= ect I could reproduce it on QEMU if I could figure out a way to configure a nvme drive that SeaBIOS/OVMF could boot off and FreeBSD 13/14 recognised. I can= get either one of those working, but not both at the same time. Reproducing Problem 1 --------------------- This only happens when using BIOS firmware and the disk layout is as shown = in the attachment problem-1-config.txt. That is the state of play just after a new install which is an upgrade from FreeBSD 13 to 14 is completed, just be= fore the reboot onto the new temporary boot environment, akips/ROOT/1, is done. After the state shown in problem-1-screenshot.png is reached, if you "tickl= e" geom/cam by plugging in a bootable USB stick (see problem-1-screenshot-tickle.png) then type the right incantations into the "mountpoint>" prompt, zfs finds the root partition and the machine boots successfully. Once it's booted successfully the error doesn't happen any m= ore. Reproducing Problem 2 --------------------- This only happens when using EFI firmware and the disk layout below is as s= hown in the attachment problem-2-config.txt. This is the state of play just aft= er a new install which is an upgrade from FreeBSD 13 to 14 is completed. The ne= xt step is to reboot onto the new temporary boot environment, akips/ROOT/1. Other notes ----------- Both these are upgrades from a fresh install of FreeBSD 13 to 14. In both cases, if you do a fresh install of the FreeBSD 14 image onto the same hard= ware under the same conditions it works. --=20 You are receiving this mail because: You are the assignee for the bug.=