unsubscribe
Jonathan Rowley
jonathans.email at icloud.com
Sun Apr 7 17:32:00 UTC 2019
> On 7 Apr 2019, at 13:00, freebsd-virtualization-request at freebsd.org wrote:
>
> Send freebsd-virtualization mailing list submissions to
> freebsd-virtualization at freebsd.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> or, via email, send a message with subject or body 'help' to
> freebsd-virtualization-request at freebsd.org
>
> You can reach the person managing the list at
> freebsd-virtualization-owner at freebsd.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of freebsd-virtualization digest..."
>
>
> Today's Topics:
>
> 1. Re: running FreePBX SNG7 Official Distro (Victor Sudakov)
> 2. Re: running FreePBX SNG7 Official Distro (Victor Sudakov)
> 3. Re: running FreePBX SNG7 Official Distro (Rodney W. Grimes)
> 4. Re: running FreePBX SNG7 Official Distro (Rodney W. Grimes)
> 5. Re: running FreePBX SNG7 Official Distro (Victor Sudakov)
> 6. Re: running FreePBX SNG7 Official Distro (Victor Sudakov)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sun, 7 Apr 2019 09:37:43 +0700
> From: Victor Sudakov <vas at mpeks.tomsk.su>
> To: freebsd-virtualization at freebsd.org
> Subject: Re: running FreePBX SNG7 Official Distro
> Message-ID: <20190407023743.GB99339 at admin.sibptus.ru>
> Content-Type: text/plain; charset="us-ascii"
>
> Rodney W. Grimes wrote:
>>>>
>>>> You can usually use the host by doing mdconfig -f <path+to+diskimage>
>>>
>>> Unfortunately mdconfig does not work with zvols:
>>>
>>> root at vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0
>>> mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file
>>
>> If its a zvol cant you just do
>> gpart show /dev/zvol/d02/vm/freepbx/disk0
>>
>> and
>> mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2
>
> No I can't if the zvol is in the "volmode=dev" mode which is the default.
>
> This is the default for a reason: it's no good exposing scores of always
> coming and going guest geoms to the host system. I think you can even
> get a conflict of labels or something like that one day.
>
>>>>> Moreover, I waited (for a long time!) for the EFI interactive shell
>>>>> prompt and with a few commands:
>>>>
>>>> Yes, the timeout is very long, and I do not know that we
>>>> document anyplace that if you wait long enough at a failed
>>>> boot you do get a EFI shell prompt eventually.
>>>
>>> Can I press some key to escape to the EFI shell?
>> Not that I am aware of.
>
> It's a major problem! There must be a well-known way to break the boot
> sequence any time and enter the EFI shell.
>
>>>>> I can guess that it looks for a FAT16 partition in the GPT with the type
>>>>> "efi" but the rest is a mystery for me. Why is it trying to find
>>>>> "grubx64.efi" and not the default "boot64.efi" (which is present), for
>>>>> example?
>>>>
>>>> I suspect that what ever guest you installed installed something
>>>> else someplace, either within the eft partition, or possibly in
>>>> the MBR?
>>>
>>> Do you mean to say, the guest installing something else someplace can
>>> influence the boot sequence of bhyve efi?
>>
>> The guest created all of the bits on that zvol,
>> it can influence many things. There is probably a tiny initial
>> stub that efi loads that has this bath to grubx64.efi codded in
>> it and that is what is causing this issue.
>
> It is very important to find and debug it because Oracle VirtualBox in
> UEFI mode installs and runs this guest just fine. So it must be some
> issue in bhyve itself.
>
> Here is the complete archive of everything the guest created in the EFI
> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz
> can you find those confusing bits?
>
> The standard procedure should be as follows:
>
> Automated detection relies on standardized file paths to the OS
> loader, with the path varying depending on the computer architecture.
> The format of the file path is defined as
> <EFI_SYSTEM_PARTITION>/EFI/BOOT/BOOT<MACHINE_TYPE_SHORT_NAME>.EFI; for
> example, the file path to the OS loader on an x86-64 system is
> /efi/BOOT/BOOTX64.EFI and efi\boot\bootaa64.efi on ARM64 architecture.
>
> Nothing about grub*.efi. But only bhyve is confused, VirtualBox is not.
>
> --
> Victor Sudakov, VAS4-RIPE, VAS47-RIPN
> 2:5005/49 at fidonet http://vas.tomsk.ru/
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: signature.asc
> Type: application/pgp-signature
> Size: 455 bytes
> Desc: not available
> URL: <http://lists.freebsd.org/pipermail/freebsd-virtualization/attachments/20190407/aca49b08/attachment-0001.sig>
>
> ------------------------------
>
> Message: 2
> Date: Sun, 7 Apr 2019 10:12:37 +0700
> From: Victor Sudakov <vas at mpeks.tomsk.su>
> To: freebsd-virtualization at freebsd.org
> Subject: Re: running FreePBX SNG7 Official Distro
> Message-ID: <20190407031237.GA7489 at admin.sibptus.ru>
> Content-Type: text/plain; charset="us-ascii"
>
> Victor Sudakov wrote:
>>>>>> I can guess that it looks for a FAT16 partition in the GPT with the type
>>>>>> "efi" but the rest is a mystery for me. Why is it trying to find
>>>>>> "grubx64.efi" and not the default "boot64.efi" (which is present), for
>>>>>> example?
>>>>>
>>>>> I suspect that what ever guest you installed installed something
>>>>> else someplace, either within the eft partition, or possibly in
>>>>> the MBR?
>>>>
>>>> Do you mean to say, the guest installing something else someplace can
>>>> influence the boot sequence of bhyve efi?
>>>
>>> The guest created all of the bits on that zvol,
>>> it can influence many things. There is probably a tiny initial
>>> stub that efi loads that has this bath to grubx64.efi codded in
>>> it and that is what is causing this issue.
>>
>> It is very important to find and debug it because Oracle VirtualBox in
>> UEFI mode installs and runs this guest just fine. So it must be some
>> issue in bhyve itself.
>>
>> Here is the complete archive of everything the guest created in the EFI
>> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz
>> can you find those confusing bits?
>
> I got it! bhyve does the right thing: it tries to boot BOOTX64.EFI, but
> BOOTX64.EFI makes it look for grubx64.efi. So BOOTX64.EFI must be some
> kind of chain loader.
>
> Watch the interactive session below. It does not however mean that there is
> nothing to fix. As I said Oracle VirtualBox in UEFI mode installs and runs this
> guest just fine.
>
> FS0:\> cd EFI
> FS0:\EFI\> ls
> Directory of: FS0:\EFI\
> 04/04/2019 15:53 <DIR> 2,048 .
> 04/04/2019 15:53 <DIR> 0 ..
> 04/04/2019 16:26 <DIR> 2,048 centos
> 04/06/2019 04:19 <DIR> 2,048 BOOT
> 0 File(s) 0 bytes
> 4 Dir(s)
> FS0:\EFI\> cd BOOT
> FS0:\EFI\BOOT\> ls
> Directory of: FS0:\EFI\BOOT\
> 04/04/2019 16:18 <DIR> 2,048 .
> 04/04/2019 16:18 <DIR> 2,048 ..
> 08/31/2017 21:30 1,296,176 BOOTX64.EFI
> 08/31/2017 21:30 79,048 fbx64.efi
> 2 File(s) 1,375,224 bytes
> 2 Dir(s)
> FS0:\EFI\BOOT\> BOOTX64.EFI
> Failed to set MokListRT: Invalid Parameter
> Failed to open \EFI\BOOT\grubx64.efi - Not Found
> Failed to load image \EFI\BOOT\grubx64.efi: Not Found
> start_image() returned Not Found
> FS0:\EFI\BOOT\>
>
> --
> Victor Sudakov, VAS4-RIPE, VAS47-RIPN
> 2:5005/49 at fidonet http://vas.tomsk.ru/
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: signature.asc
> Type: application/pgp-signature
> Size: 455 bytes
> Desc: not available
> URL: <http://lists.freebsd.org/pipermail/freebsd-virtualization/attachments/20190407/92ff99aa/attachment-0001.sig>
>
> ------------------------------
>
> Message: 3
> Date: Sat, 6 Apr 2019 21:19:37 -0700 (PDT)
> From: "Rodney W. Grimes" <freebsd-rwg at gndrsh.dnsmgr.net>
> To: Victor Sudakov <vas at mpeks.tomsk.su>
> Cc: freebsd-virtualization at freebsd.org
> Subject: Re: running FreePBX SNG7 Official Distro
> Message-ID: <201904070419.x374JbIp048373 at gndrsh.dnsmgr.net>
> Content-Type: text/plain; charset=US-ASCII
>
>> Rodney W. Grimes wrote:
>>>>>
>>>>> You can usually use the host by doing mdconfig -f <path+to+diskimage>
>>>>
>>>> Unfortunately mdconfig does not work with zvols:
>>>>
>>>> root at vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0
>>>> mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file
>>>
>>> If its a zvol cant you just do
>>> gpart show /dev/zvol/d02/vm/freepbx/disk0
>>>
>>> and
>>> mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2
>>
>> No I can't if the zvol is in the "volmode=dev" mode which is the default.
>>
>> This is the default for a reason: it's no good exposing scores of always
>> coming and going guest geoms to the host system. I think you can even
>> get a conflict of labels or something like that one day.
>
> So it may take a few more commands but it should be
> possible to do this from the host side using host
> side tools without having to boot a guest to make
> these corrections.
>
>>>>>> Moreover, I waited (for a long time!) for the EFI interactive shell
>>>>>> prompt and with a few commands:
>>>>>
>>>>> Yes, the timeout is very long, and I do not know that we
>>>>> document anyplace that if you wait long enough at a failed
>>>>> boot you do get a EFI shell prompt eventually.
>>>>
>>>> Can I press some key to escape to the EFI shell?
>>> Not that I am aware of.
>>
>> It's a major problem! There must be a well-known way to break the boot
>> sequence any time and enter the EFI shell.
>
> Agreed, hopefully those working on edk2 take note and either
> chime in with what that way is, or create a bug and track
> so that someone may fix this issue.
>
>>>>>> I can guess that it looks for a FAT16 partition in the GPT with the type
>>>>>> "efi" but the rest is a mystery for me. Why is it trying to find
>>>>>> "grubx64.efi" and not the default "boot64.efi" (which is present), for
>>>>>> example?
>>>>>
>>>>> I suspect that what ever guest you installed installed something
>>>>> else someplace, either within the eft partition, or possibly in
>>>>> the MBR?
>>>>
>>>> Do you mean to say, the guest installing something else someplace can
>>>> influence the boot sequence of bhyve efi?
>>>
>>> The guest created all of the bits on that zvol,
>>> it can influence many things. There is probably a tiny initial
>>> stub that efi loads that has this bath to grubx64.efi codded in
>>> it and that is what is causing this issue.
>>
>> It is very important to find and debug it because Oracle VirtualBox in
>> UEFI mode installs and runs this guest just fine. So it must be some
>> issue in bhyve itself.
>
> As I stated earlier bhyve is missing percistant efi variables,
> and that is most likely the reason that VirtualBox just works
> and bhyve does not.
>
>> Here is the complete archive of everything the guest created in the EFI
>> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz
>> can you find those confusing bits?
>
> Probably you well find in your VirtualBox directory a
> file that is used to store efivars, that is where the
> difference occurs.
>
>> The standard procedure should be as follows:
>>
>> Automated detection relies on standardized file paths to the OS
>> loader, with the path varying depending on the computer architecture.
>> The format of the file path is defined as
>> <EFI_SYSTEM_PARTITION>/EFI/BOOT/BOOT<MACHINE_TYPE_SHORT_NAME>.EFI; for
>> example, the file path to the OS loader on an x86-64 system is
>> /efi/BOOT/BOOTX64.EFI and efi\boot\bootaa64.efi on ARM64 architecture.
>>
>> Nothing about grub*.efi. But only bhyve is confused, VirtualBox is not.
>
> bhyve is not really confused, it is simply missing a feature
> that this guest depends on for its boot procedures. This is
> a well known miss-feature, but I do not know of anyone actively
> working on fixing it.
>
>> Victor Sudakov, VAS4-RIPE, VAS47-RIPN
>> 2:5005/49 at fidonet http://vas.tomsk.ru/
> --
> Rod Grimes rgrimes at freebsd.org
>
>
> ------------------------------
>
> Message: 4
> Date: Sat, 6 Apr 2019 21:26:06 -0700 (PDT)
> From: "Rodney W. Grimes" <freebsd-rwg at gndrsh.dnsmgr.net>
> To: Victor Sudakov <vas at mpeks.tomsk.su>
> Cc: freebsd-virtualization at freebsd.org
> Subject: Re: running FreePBX SNG7 Official Distro
> Message-ID: <201904070426.x374Q641048406 at gndrsh.dnsmgr.net>
> Content-Type: text/plain; charset=US-ASCII
>
>> Victor Sudakov wrote:
>>>>>>> I can guess that it looks for a FAT16 partition in the GPT with the type
>>>>>>> "efi" but the rest is a mystery for me. Why is it trying to find
>>>>>>> "grubx64.efi" and not the default "boot64.efi" (which is present), for
>>>>>>> example?
>>>>>>
>>>>>> I suspect that what ever guest you installed installed something
>>>>>> else someplace, either within the eft partition, or possibly in
>>>>>> the MBR?
>>>>>
>>>>> Do you mean to say, the guest installing something else someplace can
>>>>> influence the boot sequence of bhyve efi?
>>>>
>>>> The guest created all of the bits on that zvol,
>>>> it can influence many things. There is probably a tiny initial
>>>> stub that efi loads that has this bath to grubx64.efi codded in
>>>> it and that is what is causing this issue.
>>>
>>> It is very important to find and debug it because Oracle VirtualBox in
>>> UEFI mode installs and runs this guest just fine. So it must be some
>>> issue in bhyve itself.
>>>
>>> Here is the complete archive of everything the guest created in the EFI
>>> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz
>>> can you find those confusing bits?
>>
>> I got it! bhyve does the right thing: it tries to boot BOOTX64.EFI, but
>> BOOTX64.EFI makes it look for grubx64.efi. So BOOTX64.EFI must be some
>> kind of chain loader.
>
> And it brobably tries to read a efivariable, and if that variable
> is not set it defaults to grubx64.efi. This bootx64.efi is something
> the guest installed into the EFI partition, hence my assertion that
> the issue is with something the guest installed is some what valid.
>
> There are some 3rd party EFI boot managers that might help
> resolve this problem, or simply use the work around that
> I provided earlier until we can get efivars working in
> bhyve.
>
>
>> Watch the interactive session below. It does not however mean that there is
>> nothing to fix. As I said Oracle VirtualBox in UEFI mode installs and runs this
>> guest just fine.
>>
>> FS0:\> cd EFI
>> FS0:\EFI\> ls
>> Directory of: FS0:\EFI\
>> 04/04/2019 15:53 <DIR> 2,048 .
>> 04/04/2019 15:53 <DIR> 0 ..
>> 04/04/2019 16:26 <DIR> 2,048 centos
>> 04/06/2019 04:19 <DIR> 2,048 BOOT
>> 0 File(s) 0 bytes
>> 4 Dir(s)
>> FS0:\EFI\> cd BOOT
>> FS0:\EFI\BOOT\> ls
>> Directory of: FS0:\EFI\BOOT\
>> 04/04/2019 16:18 <DIR> 2,048 .
>> 04/04/2019 16:18 <DIR> 2,048 ..
>> 08/31/2017 21:30 1,296,176 BOOTX64.EFI
>> 08/31/2017 21:30 79,048 fbx64.efi
>> 2 File(s) 1,375,224 bytes
>> 2 Dir(s)
>> FS0:\EFI\BOOT\> BOOTX64.EFI
>> Failed to set MokListRT: Invalid Parameter
>> Failed to open \EFI\BOOT\grubx64.efi - Not Found
>> Failed to load image \EFI\BOOT\grubx64.efi: Not Found
>> start_image() returned Not Found
>> FS0:\EFI\BOOT\>
>>
>> --
>> Victor Sudakov, VAS4-RIPE, VAS47-RIPN
>> 2:5005/49 at fidonet http://vas.tomsk.ru/
>
> --
> Rod Grimes rgrimes at freebsd.org
>
>
> ------------------------------
>
> Message: 5
> Date: Sun, 7 Apr 2019 15:02:35 +0700
> From: Victor Sudakov <vas at mpeks.tomsk.su>
> To: freebsd-virtualization at freebsd.org
> Subject: Re: running FreePBX SNG7 Official Distro
> Message-ID: <20190407080235.GA40361 at admin.sibptus.ru>
> Content-Type: text/plain; charset="us-ascii"
>
> Rodney W. Grimes wrote:
>>>>>>
>>>>>> You can usually use the host by doing mdconfig -f <path+to+diskimage>
>>>>>
>>>>> Unfortunately mdconfig does not work with zvols:
>>>>>
>>>>> root at vas:~ # mdconfig -a -f /dev/zvol/d02/vm/freepbx/disk0
>>>>> mdconfig: /dev/zvol/d02/vm/freepbx/disk0 is not a regular file
>>>>
>>>> If its a zvol cant you just do
>>>> gpart show /dev/zvol/d02/vm/freepbx/disk0
>>>>
>>>> and
>>>> mount -t msdosfs /dev/zvol/d02/vm/freepbx/disk0p2
>>>
>>> No I can't if the zvol is in the "volmode=dev" mode which is the default.
>>>
>>> This is the default for a reason: it's no good exposing scores of always
>>> coming and going guest geoms to the host system. I think you can even
>>> get a conflict of labels or something like that one day.
>>
>> So it may take a few more commands but it should be
>> possible to do this from the host side using host
>> side tools without having to boot a guest to make
>> these corrections.
>
> I'm not aware of such commands. If anyone knows them please share with us.
>
> Moreover, I already asked a similar question in February under the
> topic "mounting/exporting/importing a zfs volume" and nobody gave a
> recipe.
>
> --
> Victor Sudakov, VAS4-RIPE, VAS47-RIPN
> 2:5005/49 at fidonet http://vas.tomsk.ru/
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: signature.asc
> Type: application/pgp-signature
> Size: 455 bytes
> Desc: not available
> URL: <http://lists.freebsd.org/pipermail/freebsd-virtualization/attachments/20190407/17de57bb/attachment-0001.sig>
>
> ------------------------------
>
> Message: 6
> Date: Sun, 7 Apr 2019 15:14:45 +0700
> From: Victor Sudakov <vas at mpeks.tomsk.su>
> To: freebsd-virtualization at freebsd.org
> Subject: Re: running FreePBX SNG7 Official Distro
> Message-ID: <20190407081445.GB40361 at admin.sibptus.ru>
> Content-Type: text/plain; charset="us-ascii"
>
> Rodney W. Grimes wrote:
>>>>>>>> I can guess that it looks for a FAT16 partition in the GPT with the type
>>>>>>>> "efi" but the rest is a mystery for me. Why is it trying to find
>>>>>>>> "grubx64.efi" and not the default "boot64.efi" (which is present), for
>>>>>>>> example?
>>>>>>>
>>>>>>> I suspect that what ever guest you installed installed something
>>>>>>> else someplace, either within the eft partition, or possibly in
>>>>>>> the MBR?
>>>>>>
>>>>>> Do you mean to say, the guest installing something else someplace can
>>>>>> influence the boot sequence of bhyve efi?
>>>>>
>>>>> The guest created all of the bits on that zvol,
>>>>> it can influence many things. There is probably a tiny initial
>>>>> stub that efi loads that has this bath to grubx64.efi codded in
>>>>> it and that is what is causing this issue.
>>>>
>>>> It is very important to find and debug it because Oracle VirtualBox in
>>>> UEFI mode installs and runs this guest just fine. So it must be some
>>>> issue in bhyve itself.
>>>>
>>>> Here is the complete archive of everything the guest created in the EFI
>>>> partition: http://admin.sibptus.ru/~vas/freepbx.tar.gz
>>>> can you find those confusing bits?
>>>
>>> I got it! bhyve does the right thing: it tries to boot BOOTX64.EFI, but
>>> BOOTX64.EFI makes it look for grubx64.efi. So BOOTX64.EFI must be some
>>> kind of chain loader.
>>
>> And it brobably tries to read a efivariable, and if that variable
>> is not set it defaults to grubx64.efi. This bootx64.efi is something
>> the guest installed into the EFI partition, hence my assertion that
>> the issue is with something the guest installed is some what valid.
>
> Do you think the guest OS installer set some efi variable during the
> installation process, which bhyve did not save? That would explain a
> lot.
>
>>>>>>> Moreover, I waited (for a long time!) for the EFI interactive shell
>>>>>>> prompt and with a few commands:
>>>>>>
>>>>>> Yes, the timeout is very long, and I do not know that we
>>>>>> document anyplace that if you wait long enough at a failed
>>>>>> boot you do get a EFI shell prompt eventually.
>>>>>
>>>>> Can I press some key to escape to the EFI shell?
>>>> Not that I am aware of.
>>>
>>> It's a major problem! There must be a well-known way to break the boot
>>> sequence any time and enter the EFI shell.
>>
>> Agreed, hopefully those working on edk2 take note and either
>> chime in with what that way is, or create a bug and track
>> so that someone may fix this issue.
>
> Would it be useful to create a PR in the FreeBSD bugtracker with a
> feature request?
>
>>>
>>> It is very important to find and debug it because Oracle VirtualBox in
>>> UEFI mode installs and runs this guest just fine. So it must be some
>>> issue in bhyve itself.
>>
>> As I stated earlier bhyve is missing percistant efi variables,
>> and that is most likely the reason that VirtualBox just works
>> and bhyve does not.
>>
>> Probably you well find in your VirtualBox directory a
>> file that is used to store efivars, that is where the
>
> I'll look into the VirtualBox directory tomorrow and report here. I was
> under the impression that efivars are stored in a configuration file in
> the EFI partition but I was probably wrong, they are kept in NVRAM
> somewhere, like BIOS settings, and not on a disk.
>
> --
> Victor Sudakov, VAS4-RIPE, VAS47-RIPN
> 2:5005/49 at fidonet http://vas.tomsk.ru/
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: signature.asc
> Type: application/pgp-signature
> Size: 455 bytes
> Desc: not available
> URL: <http://lists.freebsd.org/pipermail/freebsd-virtualization/attachments/20190407/c0afc2a8/attachment-0001.sig>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> freebsd-virtualization at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe at freebsd.org"
>
>
> ------------------------------
>
> End of freebsd-virtualization Digest, Vol 436, Issue 7
> ******************************************************
More information about the freebsd-virtualization
mailing list