Re: kernel update broke -current

Van: Ronald Klop <ronald-lists@klop.ws>
Datum: 6 september 2022 23:42
Aan: freebsd-arm <freebsd-arm@freebsd.org>, Warner Losh <imp@bsdimp.com>, Mark Millard <marklmi@yahoo.com>
Onderwerp: Re: kernel update broke -current

> 
> 
> 
> 
> 
> Van: Warner Losh <imp@bsdimp.com>
> Datum: 6 september 2022 18:13
> Aan: Mark Millard <marklmi@yahoo.com>
> CC: freebsd-arm <freebsd-arm@freebsd.org>
> Onderwerp: Re: kernel update broke -current
> 
>> 
>> 
>> 
>> On Mon, Sep 5, 2022 at 4:21 PM Mark Millard <marklmi@yahoo.com> wrote:
>> 
>>> Warner Losh <imp_at_bsdimp.com> wrote on
>>> Date: Mon, 05 Sep 2022 20:07:33 UTC :
>>> 
>>> > On Sun, Sep 4, 2022 at 4:51 PM void <void@f-m.fm> wrote:
>>> > 
>>> > > On Sun, Sep 04, 2022 at 07:31:59PM +0000, void wrote:
>>> > > >
>>> > > >On Sun, 4 Sep 2022, at 19:07, Warner Losh wrote:
>>> > > >> You need a newer boot loader...
>>> > > >
>>> > > >I was thinking - getting latest -current image, booting to it then
>>> > > >copying the contents of /boot from the image onto the mounted zpool ...
>>> > > >
>>> > > >feasible?
>>> > >
>>> > > Seems only EFI/ needed replacing from a recent snapshot. I thought it might
>>> > > be all of /boot but I was wrong. Thank you Mark!
>>> > >
>>> > 
>>> > Yes. You'll need to update EFI/BOOT on your ESP. The rest of /boot is
>>> > updated
>>> > when you do an installworld.
>>> 
>>> One of the oddities of the update sequence instructions is
>>> the lack of coverage of the likes of:
>>> 
>>> Load Path: /efiootootaa64.efi
>>> 
>>> What step of the sequencing for the overall system update?
>>> When is such an update required? (Here the example would
>>> be: Before rebooting when the ZFS pool(s) possibly used to
>>> boot gain new features?) When is it not required to update
>>> the loader in the ESP (or analogous)? Even just knowing
>>> the stage at which one should do the update indicates some
>>> about when to figure out if an update is needed and so
>>> prompts to be ready.
>> 
>> 
>> Today:
>> 
>> make installworld installkernel
>> mount -t msdos /dev/<mumble-esp> /boot/efi
>> 
>> and then one of three scenarios
>> 
>> (1) You have the old boot1.efi. This will *always* be in ESP:EFIBOOTBOOT${ARCH}.EFI.
>> (see uefi(8) for the values of ARCH). You can automatically detect this for most installations
>> that were done since about FreeBSD 12:
>> % sudo efivar | grep Boot1
>> cfee69ad-a0de-47a9-93a8-f63106f8ae99-Boot1Path
>> cfee69ad-a0de-47a9-93a8-f63106f8ae99-Boot1Dev
>> 
>> 
>> In this case you would do:
>> 
>> % sudo cp /boot/boot1.efi /boot/efi/efi/boot/boot${ARCH}.efi and you are done.
>> 
>> Please note: If your system was installed a long time ago, you should also check to see if
>> you have a LoaderPath variable set:
>> % sudo efivar --utf cfee69ad-a0de-47a9-93a8-f63106f8ae99-LoaderPath
>> 
>> /boot/loader.efi
>> 
>> When Boot1 isn't set and it is set to the above value (or something similar), then you are
>> booting with a really old copy of boot1.efi. You will need to update it and may need to arrange
>> for a larger ESP since FreeBSD's boot1.efifat was a tiny image that was hard to grow. Steal
>> space from swap for a larger ESP, etc. Documenting that is beyond the scope of this email.
>> 
>> (2) You are booting with loader.efi and it is in the bug-compatibility place. Some UEFI BIOSes
>> don't respect or can't set BootXXXX variables set by efibootmgr(8). In that case, many people
>> are booting from the 'compatibiltiy' location for removable devices. This will almost always be
>> a single boot scenario because you can't easily boot other systems like this (well, unless 'quit'
>> works from the OK prompt to go to the next one on the list, but I digress). You will see something like:
>> 
>> % sudo efivar --utf cfee69ad-a0de-47a9-93a8-f63106f8ae99-LoaderPath
>> cfee69ad-a0de-47a9-93a8-f63106f8ae99-LoaderPath
>> EFIBOOTBOOTX64.EFI
>> (or AA64)
>> 
>> when you are booting this way. In this case the update command is:
>> 
>> % sudo cp /boot/loader.efi /boot/efi/efi/boot/boot${ARCH}.efi
>> 
>> to upgrade. Some people may be doing this already on systems that can support efibootmgr(8) and
>> they can of course upgrade to using that, though that's also beyond the scope of this email.
>> 
>> (3) You are booting on a working system with a FreeBSD installed by the boot loader, or some other
>> custom arrangement. In that case, you'll see something like:
>> sudo efivar --utf cfee69ad-a0de-47a9-93a8-f63106f8ae99-LoaderPath
>> cfee69ad-a0de-47a9-93a8-f63106f8ae99-LoaderPath
>> EFIFREEBSDLOADER.EFI
>> 
>> (or some other place for custom setups: I assume if you are doing that you know enough to
>> update my instructions as appropriate). To update, you'd do
>> 
>> % sudo cp /boot/loader.efi /boot/efi/efi/freebsd/loader.efi
>> 
>> Automating all these cases is, needless to say, tricky.
>> 
>>  
>>> Part of the sequencing gets into having needed to do a
>>> installworld to have a from-same-build content replacement
>>> for the likes of /efiootootaa64.efi . So installkernel
>>> already done, a reboot already done, and an installworld
>>> having been done as well in order to have something to
>>> copy over. (There are no pointers to alternate places to
>>> get copies that I know of. One can find copies in the
>>> build tree when one builds from source locally. So waiting
>>> is not really required for that context.)
>> 
>> 
>> Yea, I have a branch that has an 'installboot' target that updates the boot
>> loader regardless, but given the 50-odd combinations of ways we can
>> boot, it's a bit bogged down.
>>  
>>> This also make it seem that updating ZFS pool features
>>> should wait until after a system upgrade that spans both
>>> the loader and kernel being ready for the new features,
>>> even if compatibility with other systems is not a worry.
>> 
>> 
>> Yes. ALWAYS update your boot blocks before zpool update.
>>  
>>> Do any of the system upgrade instructions cover such
>>> relationships? Do any of the ZFS pool upgrade instructions
>>> cover such? Does zpool or the like suggest such issues
>>> when it reports there are new features that could be
>>> enabled?
>> 
>> 
>> See above. :)
>>  
>>> Part of what I expect happened here was contributed to by
>>> a lack of being prompted to even think about the relevant
>>> issues, leading to a pool feature upgrade that had not
>>> been prepared for.
>> 
>> 
>> Yea, so far 100% of the 'help, I upgraded and now the boot loader
>> can't see zfs pool' issues have been 'I didn't upgrade my loader.efi
>> properly after zpool upgrade.' It was mandatory when we had the
>> OpenZFS rebase, but it is also necessary every few OpenZFS
>> updates from upstream as nnew features are enabled.
>> 
>> I'd love for someone to add this information to the handbook, and I'd
>> happily review such an effort. I have time to do a brain dump, but not
>> to make it pretty / formatted for asciidoctor and won't for some time.
>> 
>> Warner
>>  
>>> ===
>>> Mark Millard
>>> marklmi at yahoo.com
>>> 
>> 
> 
> 
> Interesting. 
> Does the loader pass a hint to the kernel about which loader was used to load the kernel?
> 
> Regards,
> Ronald
> 
> 

Oh I pressed send too soon. I now understand all the tools you already mentioned.
Sorry for the noise. 

Regards