From nobody Tue Sep 07 09:34:13 2021 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 7B06317A7339 for ; Tue, 7 Sep 2021 09:34:17 +0000 (UTC) (envelope-from gardask@gmail.com) Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H3g890frvz4snq for ; Tue, 7 Sep 2021 09:34:17 +0000 (UTC) (envelope-from gardask@gmail.com) Received: by mail-ej1-x62b.google.com with SMTP id bt14so18456067ejb.3 for ; Tue, 07 Sep 2021 02:34:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=0YU2jwlMYChcmCwWFsR+gQm8O5ftYkSf15drpaKcE0s=; b=dVBs5x7/0sBskbs6M8+mSoT9a5upkIMeBBO74+qkZ/3OcA1JkudA3w6qu7F9AyLgnE NbhbM61G0I1PKH7J9l/KQ9Hlz6gxofDHBzeijkfWVHLh9RaS9SkxZFpgbH4KmEHpm2WQ zo3R4Si3JTEaUViG3+BbiL5QkxYNP3AW3n0ZwMD9CCCDZLT4nCOrPn2fAnD6nfFzhu1V EfLIoADxIl1rUhagtTkI2bZVvtbuCuWo8vIsHIgGFcdNz0oZIYyIqBQC7y2rnTanzAbw PJd9yNVZIwp8r67Xlg7lIJacRX0sCv8Eou1ezXU6w0sC+Q1DCiYFV7XHkvy93kvNiWvC 8W+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=0YU2jwlMYChcmCwWFsR+gQm8O5ftYkSf15drpaKcE0s=; b=KYJREH0ALyBCV2ouw1fJYeJOlElPvuGq+xoXCK8/n0LsfVGDxF6jrtp1xTau161Rrf cFKHP951QIJIIX2VUDCdu9IG0g1OmiI5xStJl4YLdfFDFzF/Za0i7WAKEI1LiFBe6sDM 88vSjd5OezguQDimWTTg7TE6bwFpFLfZIHD3rqCklIu+iklToDNsTQQ7jdAqXWaOusMI FeN/f0h9J0jCZ4hb7OEH22bOjnElJ2AuUhFFc3V6fQ0dOmA54XPPuelfqV4f7IMuVQPv chQMuT0vHDz85mp6VrWgtAR8nw1+fqoj3M3G8xp4TkllJ9gL5Re5g738nFcfaWqliYvs WHSw== X-Gm-Message-State: AOAM5309gnXtLcZMe4E/vtr9Jac3CUh5S8RKJ9wj8g0PmvZU2mSDhIS2 yxxsxD+Yv/ZedxpZH/eG4qmCAQIm0rU= X-Google-Smtp-Source: ABdhPJyN7gBNog32d5AU5Gm9l+9Czn4RoTJNrPDDEE/bfRQ2GafwvC2WvHCGcTBHfXovGtqnp8l6AA== X-Received: by 2002:a17:906:6808:: with SMTP id k8mr18226667ejr.138.1631007254586; Tue, 07 Sep 2021 02:34:14 -0700 (PDT) Received: from [10.0.30.5] ([31.47.99.1]) by smtp.gmail.com with ESMTPSA id q15sm5218763ejx.3.2021.09.07.02.34.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 07 Sep 2021 02:34:14 -0700 (PDT) Subject: Re: Install to ZFS root is using device names hence failing when device tree is changed. To: Peter Jeremy Cc: current@freebsd.org References: <73602bc2-44d9-f45e-e91e-fb0bd30dd640@gmail.com> From: Karel Gardas Message-ID: <68531cbc-87a9-7a1a-8de5-922b2402de02@gmail.com> Date: Tue, 7 Sep 2021 11:34:13 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4H3g890frvz4snq X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On 9/7/21 10:29 AM, Peter Jeremy wrote: > On 2021-Sep-06 17:45:31 +0200, Karel Gardas wrote: >> just installed 14-current snapshot from 2.9. on uefi amd64 machine. >> Installed from USB memstick which was detected as da0 into the ssd >> hanging on usb3 in external enclosure which was detected as da1. >> >> ZFS root pool is then using /dev/da1p3 as swap and /dev/da1p1 as >> /boot/efi and probably also something as root zpool. >> >> Anyway, expected thing happen. When I pulled out USB stick identified as >> da0 on reboot, the drive on USB3 switch from da1 to da0 and result is >> unbootable system with complains about various /dev/da1xx drives missing >> for swap efi boot etc. > > Can you give more details about exactly what the errors and when they > occur during the boot cycle. In particular: > * Low-level boot (anything prior to the FreeBSD kernel) knows nothing > about da0 or da1, so any problems there are associated with your > BIOS config, not FreeBSD. Low level boot is all right since kernel is booting. What's broken is root/zfs mount and swap enablement. > * The swap partition will, by default, appear as a hard-wired device > name in /etc/fstab - that will definitely need updating. This will > prevent the "swapon" working but won't prevent the boot. ACK. Good to know. > * ZFS doesn't care about device names - it looks for ZFS labels on all > possible devices. I think you are wrong here. Let's see zpool status: karel@freebsd:~ $ zpool status pool: zroot state: ONLINE config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 da1p4 ONLINE 0 0 0 errors: No known data errors Now, let's reboot and see error on serial console when install da0 is not attached: Root mount waiting for: usbus0 Root mount waiting for: usbus0 ugen0.5: at usbus0 umass0 numa-domain 0 on uhub1 umass0: on usbus0 umass0: SCSI over Bulk-Only; quirks = 0x0100 umass0:9:0: Attached to scbus9 da0 at umass-sim0 bus 0 scbus9 target 0 lun 0 da0: Fixed Direct Access SPC-4 SCSI device da0: Serial Number 2017010000C7 da0: 400.000MB/s transfers da0: 238475MB (488397168 512 byte sectors) da0: quirks=0x2 Dual Console: Serial Primary, Video Secondary No suitable dump device was found. Setting hostuuid: 8cdf33eb-6866-42ae-a49d-ae7ee4c0c33c. Setting hostid: 0xdf6467d8. no pools available to import swapon: /dev/da1p3: No such file or directory Starting file system checks: Can't open `/dev/da1p1' /dev/da1p1: UNEXPECTED INCONSISTENCY; RUN fsck_msdosfs MANUALLY. THE FOLLOWING FILE SYSTEM HAD AN UNEXPECTED INCONSISTENCY: msdosfs: /dev/da1p1 (/boot/efi) Automatic file system check failed; help! ERROR: ABORTING BOOT (sending SIGTERM to parent)! 2021-09-07T11:23:23.710549+02:00 - init 1 - - /bin/sh on /etc/rc terminated abnormally, going to single user mode Enter full pathname of shell or RETURN for /bin/sh: So this is problematic /efi parition, if I remove it from the /etc/fstab I get this boot: .... Wow! It boots well, so you were indeed right. Checking zpool status reveals: karel@freebsd:~ $ zpool status pool: zroot state: ONLINE config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 da0p4 ONLINE 0 0 0 errors: No known data errors karel@freebsd:~ $ so indeed, ZFS on FreeBSD is similar to this on Solaris. On Linux I got different experience (e.g. /dev/sdaX hardwired and failing to boot) hence I've been in impression that this is also a case of FreeBSD when code is shared thorough OpenZFS project. Great to know and thanks for kicking me into it. So just /efi partition mount is the culprit here... Thanks! Karel