Re: I am sick and tired of the poor quality of documentation on FreeBSD

On 11/21/23 21:49, David Christensen wrote:
> On 11/21/23 11:51, iio7@tutanota.com wrote:
>> ... upgrade ...
>>    Pool 'zroot' has the bootfs property set, you might need to update 
>> the boot code.
>>    See gptzfsboot(8) and loader.efi(8) for details.
>>
>> I might need to update the boot code!? WTF does that even mean?
>>
>>   I never touched any boot code during installation so I don't even 
>> know what that is! And reading those man pages did NOT help me in any 
>> way.
>>
>> So now I got a box that's borked with this beautiful message after 
>> updating the zroot pool:
>>
>> ZFS: unsupported feature: com.delphix:head_errlog
>> ZFS: pool zroot is not supported
>> Can't find /boot/zfsloader
>> Can't find /boot/loader
>> Can't find /boot/kernel/kernel
> 
> 
> On 11/21/23 13:10, iio7@tutanota.com wrote:
>  > ...
>  > 
> https://www.daemonology.net/blog/2023-11-21-late-breaking-FreeBSD-14-breakage.html
>  > ...
> 
> 
> On 11/21/23 15:22, iio7@tutanota.com wrote:
>  > Enough with the assumptions!
>  > ...
> 
> 
> I upgraded a ZFS boot pool and ZFS root pool last year, and encountered 
> a similar warning:
> 
> https://lists.freebsd.org/archives/freebsd-questions/2022-July/001593.html
> 
> 
> The following article was helpful for understanding the FreeBSD boot 
> process:
> 
> https://klarasystems.com/articles/the-freebsd-boot-process/
> 
> 
> It might be possible to fix your FreeBSD instance:
> 
> 1.  Study the klarasystems article and determine the boot loader 
> sequence for your computer, what stages/ files go where, and why.
> 
> 2.  Boot FreeBSD installer media into a rescue shell.
> 
> 3.  Crawl the FreeBSD installer shell script, following the path you 
> would take if you were installing FreeBSD on that computer.  Find the 
> sections of installer code that write boot loader files from the 
> installer media to various blocks on the system disk -- MBR, space 
> between MBR and first slice, slices, BSD partition(s), etc..  Compare 
> the installer files against disk blocks using cmp(1).  If you find 
> differences, write the installer files to the appropriate system disk 
> blocks with dd(1).

Great to learn how things work, but unnecessary to fix the boot issue. 
It is either copy the working file to the efi partition or write 
gptzfsboot to the disk with gpart bootcode command.

> 4.  If and when you get to installer files copied into a system disk ZFS 
> file system, stop and see if the previous steps fixed the boot issue.
> 
> 5.  To access the system disk ZFS file systems, import the system disk 
> pool(s) using the '-R altroot' option to avoid confusion due to overlaid 
> file systems.

Don't need to access ZFS file systems on the disk during this repair; 
the code read for booting happens before ZFS is understood and so will 
be on disk in places outside of ZFS.

> I suggest taking an image of your system disk before attempting any of 
> the above.

If there is ever doubt on what will happen next on a system, this is 
always the right approach. If there is a question to the filesystem 
integrity, clone all disk sectors as is and you know you can get back to 
where you are now as long as there isn't hardware failure involved.

> Alternatively, remove the system disk, install a zeroed disk, do a fresh 
> install, and restore from backups.

Unnecessary in the described scenario, but it certainly would fix the issue.

> Beware of installing multiple disks with identically named ZFS pools -- 
> e.g. two system disks with pools "bootpool" and "zroot".  It is safer to 
> use the FreeBSD installer if you need to work on a ZFS system disk.

Another fun gotcha is if you use zpool send+zpool receive to write a 
backup of a boot disk to removable media instead of writing the 
datastream as a file to the backup media, you get to learn about the fun 
of when your backup mounted in place of the live system because its 
properties said for it to be mounted there. I make modifications to zfs 
properties on the receiving side to fix that and some other 
issues+desires but suspect there are more things I have to learn in this 
space.

> David
> 
>