Segmentation fault in grub-bhyve when trying to boot a Linux guest

Tue May 1 16:41:56 UTC 2018

On 2018-05-01T00:56:15 +0200
"Fabian Freyer" <fabian.freyer at physik.tu-berlin.de> wrote:

> On 1 May 2018, at 0:05, Mark Raynsford via freebsd-virtualization wrote:
> > I've recently attempted to install a Debian 9.4.0 x86_64 guest. The
> > installer ran to completion without issue, and I then rebooted into the
> > installed system, again without issue.
> >
> > I then shut the system down and tried to bring it up...
> >
> >   pid 71802 (grub-bhyve), uid 0: exited on signal 11 (core dumped)  
> 
> Is this reproducible? If yes,
> * is it still reproducible on a freshly built grub-bhyve from ports with
>   debugging symbols (build the port with WITH_DEBUG=yes)?
> * is a core file dumped?
> * could you grab a backtrace from the core file?

Hello!

This is still reproducible, yes. I've recompiled the port with
WITH_DEBUG=yes (which, unfortunately, took most of the day due to
having to compile the gcc-6 dependency). Unfortunately, this didn't
yield a usable backtrace. I'm guessing that the Dwarf debugging
information isn't compatible with the gdb version that's in base? gdb
says:

# gdb /usr/local/sbin/grub-bhyve grub-bhyve.core 
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...Dwarf Error: wrong version in compilation unit header (is 4, should be 2) [in module /usr/local/sbin/grub-bhyve]

Core was generated by `/usr/local/sbin/grub-bhyve -m /storage/vm/rosemary/device.map -r host -d /storag'.
Program terminated with signal 11, Segmentation fault.
Reading symbols from /lib/libncurses.so.8...(no debugging symbols found)...done.
Loaded symbols for /lib/libncurses.so.8
Reading symbols from /lib/libzfs.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libzfs.so.2
Reading symbols from /lib/libnvpair.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnvpair.so.2
Reading symbols from /lib/libgeom.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib/libgeom.so.5
Reading symbols from /usr/lib/libvmmapi.so.5...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libvmmapi.so.5
Reading symbols from /lib/libutil.so.9...(no debugging symbols found)...done.
Loaded symbols for /lib/libutil.so.9
Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.7
Reading symbols from /lib/libmd.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libmd.so.6
Reading symbols from /lib/libumem.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libumem.so.2
Reading symbols from /lib/libuutil.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libuutil.so.2
Reading symbols from /lib/libm.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib/libm.so.5
Reading symbols from /lib/libavl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libavl.so.2
Reading symbols from /lib/libbsdxml.so.4...(no debugging symbols found)...done.
Loaded symbols for /lib/libbsdxml.so.4
Reading symbols from /lib/libz.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libz.so.6
Reading symbols from /lib/libzfs_core.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libzfs_core.so.2
Reading symbols from /lib/libthr.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib/libthr.so.3
Reading symbols from /lib/libsbuf.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libsbuf.so.6
Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done.
Loaded symbols for /libexec/ld-elf.so.1
#0  0x000000000040d506 in ?? ()
(gdb) bt
#0  0x000000000040d506 in ?? ()
#1  0x00007fffffffe620 in ?? ()
#2  0x000000000049e9eb in ?? ()
#3  0x00007fffffffe620 in ?? ()
#4  0x0000000000000800 in ?? ()
#5  0x0000000000000000 in ?? ()

In the hope that you or someone else can reproduce this, or even get a 
better trace out of the core file, I've uploaded:

  * grub-bhyve.core.lzma (the LZMA compressed core file)
  * grub-bhyve.lzma (the executable that the port produced)
  * rosemary_disk0.lzma (the LZMA compressed zvol)
  * checksum.s256 (SHA256 checksums for all of the above)

https://drive.google.com/drive/folders/1hxfRqS1b0HYpcN3sglJ_To0fWG22Kjad?usp=sharing

Assuming that you want the zvol to be placed at /x/y/z/disk0, I 
think you should be able to:

  lzma -d < rosemary_disk0.lzma | zfs recv /x/y/z/disk0

And then:

  grub-bhyve \
    -m /path/to/config/dir/device.map \
    -r host \
    -d /path/to/config/dir \
    -c /dev/nmdm56A \
    -M 512M \
    rosemary

Where /path/to/config/dir/grub.cfg reads:

  linux (hd0,msdos1)/vmlinuz root=/dev/sda1
  initrd (hd0,msdos1)/initrd.img
  boot

And /path/to/config/dir/device.map reads:

  (hd0) /x/y/z/disk0

This install of FreeBSD is barely a week old, and it's running on 
a Dell PowerEdge R410 V2 with no customization on my part. I'm running
the vanilla 11.1-RELEASE-p9 binaries and haven't built anything myself.
I'd hope it should be fairly easy to reproduce this given the same ZFS
filesystem and so on...

-- 
Mark Raynsford | http://www.io7m.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 228 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-virtualization/attachments/20180501/de40cda5/attachment.sig>