Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI

Fri Mar 23 13:29:28 UTC 2018

-----Original Message-----
From: owner-freebsd-virtualization at freebsd.org <owner-freebsd-virtualization at freebsd.org> On Behalf Of Kyle Evans
Sent: 23 March 2018 13:06
To: Joe Maloney <jmaloney at ixsystems.com>
Cc: Warner Losh <imp at freebsd.org>; freebsd-virtualization at freebsd.org
Subject: Re: Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI

On Fri, Mar 23, 2018 at 3:56 AM, Joe Maloney <jmaloney at ixsystems.com> wrote:
> We narrowed the issue down to how vm-bhyve attaches a null.iso when 
> starting the VM.
>

>What exactly are the contents of this null.iso? It sounds like we're just ultimately choosing the wrong partition to boot with the null.iso attached. I'd be interested to see if a loader.efi built with >D13784 [1] applied resolves this both replacing /boot/loader.efi initially then while installed on the ESP.

>Do note that I'm pretty sure we still had some small outstanding issues with D13784's loader.efi being installed as /boot/loader.efi (with console handling), so it may not be a good long-term solution, >but this should hopefully land in the not-so-distant future.

>[1] https://reviews.freebsd.org/D13784

The original reason for null.iso was due to notes on the bhyve wiki
https://wiki.freebsd.org/bhyve/Windows

" The install.iso is only required for the first boot of Windows Server and must be removed from the bhyve command after the first boot. Desktop editions of Windows require that a null install.iso file remains and it can be created with touch install.iso"

At the time the UEFI support was primarily used for loading Windows guests, and some desktop versions apparently needed an empty iso file to boot. I was not able to verify which guests needed this file and so it was supplied to all, as it seemed to have no ill effects on the server versions of Windows I was testing with.

As mentioned above is this more an issue with the boot process? If bhyve at some point allows an "empty" cd device to be attached, with the ability to insert an iso on the fly, like many other hypervisors do, would that trigger the same issue?

Matt

> For now a hack like so to vm-bhyve can get around the issue:
>
> https://github.com/araujobsd/vm-bhyve/commit/29db2d6c6ec4a29578dc11190
> 3107f25a78cdf82
>
> This may simply be an issue with vm-bhyve attaching an invalid iso 
> image to the VM, and I would conclude it is odd that it does so.  Or 
> there may still be an issue that affects even the latest 03-22-18 
> snapshots for example if other media is present when booting with 
> bhyve, or natively.  I would need to do some more testing at a later 
> date to confirm but just wanted to pass along what was discovered to be the root cause thus far.
>
> On Friday, March 16, 2018, Marcelo Araujo <araujobsdport at gmail.com> wrote:
>>
>> 2018-03-16 9:55 GMT+08:00 Kyle Evans <kevans at freebsd.org>:
>>
>> > On Thu, Mar 15, 2018 at 8:46 PM, Marcelo Araujo 
>> > <araujobsdport at gmail.com>
>> > wrote:
>> > >
>> > >
>> > > 2018-03-16 7:07 GMT+08:00 Kyle Evans <kevans at freebsd.org>:
>> > >>
>> > >> On Thu, Mar 15, 2018 at 5:01 PM, Kyle Evans <kevans at freebsd.org>
>> > >> wrote:
>> > >> > On Thu, Mar 15, 2018 at 4:09 PM, Peter Grehan 
>> > >> > <grehan at freebsd.org>
>> > >> > wrote:
>> > >> >>> I believe the problem may have been introduced with this commit:
>> > >> >>> > >
>> > >> >>> https://svnweb.freebsd.org/base/stable/?view=log&pathrev=329
>> > >> >>> 114
>> > >> >>
>> > >> >>  Any chance of being able to work out where in that list of 
>> > >> >> commits
>> > in
>> > >> >> CURRENT the loader stopped working ?
>> > >> >>
>> > >> >
>> > >> > Indeed- if you could work out the exact commit in that range 
>> > >> > from head that caused it, I wouldn't think it to be a tough 
>> > >> > fix. After tonight I'm out until Sunday, but should have time 
>> > >> > Sunday or Monday to try and diagnose it further.
>> > >>
>> > >> Can one of you try this with boot1.efi+loader.efi built from 
>> > >> today's head stand/? I'm not sure what I'm expecting here since 
>> > >> these are among my first times trying bhyve, but this is what 
>> > >> I'm seeing now (vs. from the mentioned head snapshot where I 
>> > >> noted similar behavior as originally mentioned):
>> > >>
>> > >> 1.) Get to loader.efi, menu is good
>> > >> 2.) Break into loader prompt
>> > >> 3.) `lsdev`- pager is restricted to the line the prompt is on, 
>> > >> so the output is useless
>> > >> 4.) `boot`
>> > >> 5.) "Unhandled ps2 mouse command 0xe1"
>> > >>
>> > >> At this point, the boot looks screwed until I VNC into it- it 
>> > >> booted fine here, but the console stopped working after the kernel handoff.
>> > >>
>> > >> Thanks,
>> > >>
>> > >> Kyle Evans
>> > >> _______________________________________________
>> > >> freebsd-virtualization at freebsd.org mailing list 
>> > >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualizatio
>> > >> n To unsubscribe, send any mail to 
>> > >> "freebsd-virtualization-unsubscribe at freebsd.org"
>> > >
>> > >
>> > > Hi Kyle,
>> > >
>> > > I will do that today and report back as soon as I have something.
>> > >
>> >
>> > Thanks! If it's still failing, I think capturing the output of "lsdev"
>> > and "show currdev" prior to a failed boot might be most helpful 
>> > just to make sure there's not something obviously sketchy happening.
>> >
>>
>> Hi,
>>
>> I think we had two bad snapshots!
>>
>> I just finished the tests with HEAD and 11-STABLE latest snapshots:
>> 1) HEAD: FreeBSD-12.0-CURRENT-amd64-20180315-r331001-disc1.iso
>> 2) Stable: FreeBSD-11.1-STABLE-amd64-20180315-r330998-bootonly.iso
>>
>> I have installed it using ZFS and tested using AHCI and virtio-blk.
>>
>> So everything worked fine.
>> Seems those broken snapshots have missed some commits related with EFI.
>>
>>
>> Thank you all.
>> --
>>
>> --
>> Marcelo Araujo            (__)araujo at FreeBSD.org
>> \\\'',)http://www.FreeBSD.org <http://www.freebsd.org/>   \/  \ ^
>> Power To Server.         .\. /_)
>> _______________________________________________
>> freebsd-virtualization at freebsd.org mailing list 
>> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
>> To unsubscribe, send any mail to
>> "freebsd-virtualization-unsubscribe at freebsd.org"
>
>
>
> --
> Joe Maloney
> QA Manager / iXsystems
> Enterprise Storage & Servers Driven By Open Source
>
_______________________________________________
freebsd-virtualization at freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe at freebsd.org"