From nobody Wed Nov 22 23:16:03 2023 X-Original-To: questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SbHDc0mB9z51RDQ for ; Wed, 22 Nov 2023 23:16:16 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) Received: from holgerdanske.com (holgerdanske.com [184.105.128.27]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "holgerdanske.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SbHDZ6472z4vbn for ; Wed, 22 Nov 2023 23:16:14 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=holgerdanske.com header.s=nov-20210719-112354 header.b=AzMHMRhL; spf=pass (mx1.freebsd.org: domain of dpchrist@holgerdanske.com designates 184.105.128.27 as permitted sender) smtp.mailfrom=dpchrist@holgerdanske.com; dmarc=pass (policy=none) header.from=holgerdanske.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=holgerdanske.com; s=nov-20210719-112354; t=1700694972; bh=iPRbqJvK2Tvm/stQY0RG2IGPFJRBqDlwDQWaI4d9b8U=; h=Received:Message-ID:Date:MIME-Version:User-Agent:Subject: Content-Language:To:References:From:In-Reply-To:Content-Type: Content-Transfer-Encoding; b=AzMHMRhLDix+M5H+VDYb6iu1xfmDuHsZXSEZVouE2VRd0+aIBYJBBo2nyNdaRNxhc m5O7KDVQgS2Nisff5D6xznR488yf+UrsKCNMCw1bT4h58mu0seWnMB2vPHjLwVQ1aD nvjNnvHCICfKK8SlIF8tR8XHWOrZCFyvhdNbS0OLLnDr1eAbWuJZYBLFR2j0oegRsE Sd3vYzbKfVNJoDuVnOfe+tN1k/AK/fxAvmWL3ayCBr7CXkHODALWyLTzVawtu1NjRw cVH+zoOK2fogdRTUqqUEM7xRIJV0f4bSUJ6mXcGHOlFXNbV18Kxv9YuAhbLHcfb54t OL+qD4w4beUIiiM8136BsgQYhbG9lYk0p74uv7+kz9FSoIO5GD1a7TKAG6ZcIE7HAk z6oJOOONYSMp8A9KcdJ7jFS/BRRt9lmiH0ND+c4xLmPU5q5Odtagvb4HhbLa8RMtmQ 0hD9gHVYI1z7jNEMuPwELnbLx5EZAJf1ckwZ4I2htlBd5HZoffTYO2j3XhhFOV6YIL 8Q+SlK5KMU/R1bw/g0BwL2QVyyHWzMNkvlqzrf25RhpbUkRPi4RTCK6LR8OY77Wu6W CskgeR+cim2AMX732aaNKYN24nxzIuC0wabpofUTi1yZQRsV5K25MyaAVn6uUc7xyA U7n9ia8etrkQp0DWMgpRvuPQ= Received: from 99.100.19.101 (99-100-19-101.lightspeed.frokca.sbcglobal.net [99.100.19.101]) by holgerdanske.com with ESMTPSA (TLS_AES_128_GCM_SHA256:TLSv1.3:Kx=any:Au=any:Enc=AESGCM(128):Mac=AEAD) (SMTP-AUTH username dpchrist@holgerdanske.com, mechanism PLAIN) for ; Wed, 22 Nov 2023 15:16:12 -0800 Message-ID: <2a732fe6-9967-4c8a-93ff-552d2934ce43@holgerdanske.com> Date: Wed, 22 Nov 2023 15:16:03 -0800 List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: I am sick and tired of the poor quality of documentation on FreeBSD Content-Language: en-US To: questions@freebsd.org References: From: David Christensen In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-3.87 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.978]; DMARC_POLICY_ALLOW(-0.50)[holgerdanske.com,none]; R_DKIM_ALLOW(-0.20)[holgerdanske.com:s=nov-20210719-112354]; R_SPF_ALLOW(-0.20)[+a]; MIME_GOOD(-0.10)[text/plain]; ONCE_RECEIVED(0.10)[]; XM_UA_NO_VERSION(0.01)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[questions@freebsd.org]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_ALL(0.00)[]; ASN(0.00)[asn:6939, ipnet:184.104.0.0/15, country:US]; MLMMJ_DEST(0.00)[questions@freebsd.org]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[holgerdanske.com:+]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4SbHDZ6472z4vbn X-Spamd-Bar: --- On 11/22/23 00:07, Edward Sanford Sutton, III wrote: > On 11/21/23 21:49, David Christensen wrote: >> On 11/21/23 11:51, iio7@tutanota.com wrote: >>> ... upgrade ... >>>    Pool 'zroot' has the bootfs property set, you might need to update >>> the boot code. >>>    See gptzfsboot(8) and loader.efi(8) for details. >>> >>> I might need to update the boot code!? WTF does that even mean? >>> >>>   I never touched any boot code during installation so I don't even >>> know what that is! And reading those man pages did NOT help me in any >>> way. >>> >>> So now I got a box that's borked with this beautiful message after >>> updating the zroot pool: >>> >>> ZFS: unsupported feature: com.delphix:head_errlog >>> ZFS: pool zroot is not supported >>> Can't find /boot/zfsloader >>> Can't find /boot/loader >>> Can't find /boot/kernel/kernel >> >> >> 3.  Crawl the FreeBSD installer shell script, following the path you >> would take if you were installing FreeBSD on that computer.  Find the >> sections of installer code that write boot loader files from the >> installer media to various blocks on the system disk -- MBR, space >> between MBR and first slice, slices, BSD partition(s), etc..  Compare >> the installer files against disk blocks using cmp(1).  If you find >> differences, write the installer files to the appropriate system disk >> blocks with dd(1). > > Great to learn how things work, but unnecessary to fix the boot issue. > It is either copy the working file to the efi partition or write > gptzfsboot to the disk with gpart bootcode command. Thank you for the reply. :-) I assume those procedures can be applied to a ZFS-on-root system disk by booting FreeBSD installer media (?). Can you please provide a URL to documentation that explains the concepts and provides example commands? >> 4.  If and when you get to installer files copied into a system disk >> ZFS file system, stop and see if the previous steps fixed the boot issue. >> >> 5.  To access the system disk ZFS file systems, import the system disk >> pool(s) using the '-R altroot' option to avoid confusion due to >> overlaid file systems. > > Don't need to access ZFS file systems on the disk during this repair; > the code read for booting happens before ZFS is understood and so will > be on disk in places outside of ZFS. I suspected as much; that is why I added item #4. But, I wanted to include #5 and the "-R altroot" warning just in case any goes there. But, I forgot to mention the ESP (I use BIOS/MBR). >> I suggest taking an image of your system disk before attempting any of >> the above. > > If there is ever doubt on what will happen next on a system, this is > always the right approach. If there is a question to the filesystem > integrity, clone all disk sectors as is and you know you can get back to > where you are now as long as there isn't hardware failure involved. +1 >> Alternatively, remove the system disk, install a zeroed disk, do a >> fresh install, and restore from backups. > > Unnecessary in the described scenario, but it certainly would fix the > issue. Restoring an undamaged image is my preferred method -- the more recent the image the better. So, I take images immediately before updating/ upgrading, doing significant reconfiguration, or attempting anything risky. >> Beware of installing multiple disks with identically named ZFS pools >> -- e.g. two system disks with pools "bootpool" and "zroot".  It is >> safer to use the FreeBSD installer if you need to work on a ZFS system >> disk. > > Another fun gotcha is if you use zpool send+zpool receive to write a > backup of a boot disk to removable media instead of writing the > datastream as a file to the backup media, you get to learn about the fun > of when your backup mounted in place of the live system because its > properties said for it to be mounted there. I make modifications to zfs > properties on the receiving side to fix that and some other > issues+desires but suspect there are more things I have to learn in this > space. That sounds like cloning via replication, but with modifications in-flight (?). Working on that experiment should improve your understanding of ZFS imports and mounts, ZFS properties, and other ZFS details. David