Bhyve won't boot OpenBSD 7.4

Reply: Peter Grehan : "Re: Bhyve won't boot OpenBSD 7.4"
Go to: [ bottom of page ] [ top of archives ] [ this month ]
From: Thomas Laus <lausts_at_acm.org>
Date: Sun, 26 Nov 2023 21:10:56 UTC
I have 2 production servers that I want to upgrade to FreeBSD 14.0 that 
use Bhyve to run OpenBSD 7.3 packet packet filters and a few other 
instances.  Since this is an operating system upgrade to FreeBSD 14.0, I 
wanted to also upgrade the OpenBSD 7.3 instance to OpenBSD 7.4 that was 
released last month.  I tried the in-place OpenBSD 'sysupgrade' tool 
unsuccessfully.  This has worked for me successfully for the past 
several years.  I tried booting from an OpenBSD 7.4 CDROM to attempt the 
upgrade using this method.  It was also without success.  Thinking that 
I had a bad .iso file, I burned it to a CDROM and used it to install 
OpenBSD 7.4 on a spare drive in an old laptop.  It worked like a champ. 
I have not tried to load OpenBSD 7.4 on any other hypervisors like Vultr 
or Amazon EC2 yet to see if that version has issues with all 
hypervisors.  Here is my vm-bhyve log for the installation of OpenBSD 
7.3, the in-place upgrade to 7.4 and finally the boot attempt from the 
.iso file:

Nov 26 10:51:09: initialising
Nov 26 10:51:09:  [loader: grub]
Nov 26 10:51:09:  [cpu: 1]
Nov 26 10:51:09:  [memory: 768M]
Nov 26 10:51:09:  [hostbridge: standard]
Nov 26 10:51:09:  [com ports: com1]
Nov 26 10:51:09:  [uuid: af22d4e4-2a87-11eb-b8fd-a0369f7299dd]
Nov 26 10:51:09:  [debug mode: no]
Nov 26 10:51:09:  [primary disk: disk0.img]
Nov 26 10:51:09:  [primary disk dev: file]
Nov 26 10:51:09: initialising network device tap0
Nov 26 10:51:10: booting
Nov 26 10:51:10: create file /virtual/openbsd/device.map
Nov 26 10:51:10:  -> (cd0) /virtual/.iso/install73.iso
Nov 26 10:51:10:  -> (hd0) /virtual/openbsd/disk0.img
Nov 26 10:51:10: create file /virtual/openbsd/grub.cfg
Nov 26 10:51:10:  -> timeout=3
Nov 26 10:51:10:  -> menuentry 'openbsd (bhyve install)' {
Nov 26 10:51:10:  ->     root=cd0
Nov 26 10:51:10:  ->     kopenbsd -h com0 /7.3/amd64/bsd.rd
Nov 26 10:51:10:  -> }
Nov 26 10:51:10: /usr/local/sbin/grub-bhyve -c /dev/nmdm-openbsd.1A -S 
-m /virtual/openbsd/device.map -M 768M -r host -d /virtual/openbsd openbsd
Nov 26 10:51:13:  [bhyve options: -c 1 -m 768M -AHP -w -U 
af22d4e4-2a87-11eb-b8fd-a0369f7299dd -u -S]
Nov 26 10:51:13:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 
4:0,virtio-blk,/virtual/openbsd/disk0.img -s 
5:0,virtio-net,tap0,mac=58:9c:fc:09:93:65 -s 6:0,passthru,4/0/0]
Nov 26 10:51:13:  [bhyve console: -l com1,/dev/nmdm-openbsd.1A]
Nov 26 10:51:13:  [bhyve iso device: -s 
3:0,ahci-cd,/virtual/.iso/install73.iso,ro]
Nov 26 10:51:13: starting bhyve (run 1)
Nov 26 10:57:04: bhyve exited with status 0
Nov 26 10:57:04: restarting
Nov 26 10:57:05: create file /virtual/openbsd/device.map
Nov 26 10:57:05:  -> (hd0) /virtual/openbsd/disk0.img
Nov 26 10:57:05: create file /virtual/openbsd/grub.cfg
Nov 26 10:57:05:  -> timeout=3
Nov 26 10:57:05:  -> menuentry 'openbsd (bhyve run)' {
Nov 26 10:57:05:  ->     root=hd0,1
Nov 26 10:57:05:  ->     kopenbsd -h com0 -r sd0a /bsd
Nov 26 10:57:05:  -> }
Nov 26 10:57:05: /usr/local/sbin/grub-bhyve -c /dev/nmdm-openbsd.1A -S 
-m /virtual/openbsd/device.map -M 768M -r host -d /virtual/openbsd openbsd
Nov 26 10:57:08:  [bhyve options: -c 1 -m 768M -AHP -w -U 
af22d4e4-2a87-11eb-b8fd-a0369f7299dd -u -S]
Nov 26 10:57:08:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 
4:0,virtio-blk,/virtual/openbsd/disk0.img -s 
5:0,virtio-net,tap0,mac=58:9c:fc:09:93:65 -s 6:0,passthru,4/0/0]
Nov 26 10:57:08:  [bhyve console: -l com1,/dev/nmdm-openbsd.1A]
Nov 26 10:57:08:  [bhyve iso device: -s 
3:0,ahci-cd,/virtual/.iso/install73.iso,ro]
Nov 26 10:57:08: starting bhyve (run 2)
Nov 26 10:59:01: bhyve exited with status 0
Nov 26 10:59:01: restarting
Nov 26 10:59:02: create file /virtual/openbsd/device.map
Nov 26 10:59:02:  -> (hd0) /virtual/openbsd/disk0.img
Nov 26 10:59:02: create file /virtual/openbsd/grub.cfg
Nov 26 10:59:02:  -> timeout=3
Nov 26 10:59:02:  -> menuentry 'openbsd (bhyve run)' {
Nov 26 10:59:02:  ->     root=hd0,1
Nov 26 10:59:02:  ->     kopenbsd -h com0 -r sd0a /bsd
Nov 26 10:59:02:  -> }
Nov 26 10:59:02: /usr/local/sbin/grub-bhyve -c /dev/nmdm-openbsd.1A -S 
-m /virtual/openbsd/device.map -M 768M -r host -d /virtual/openbsd openbsd
Nov 26 10:59:06:  [bhyve options: -c 1 -m 768M -AHP -w -U 
af22d4e4-2a87-11eb-b8fd-a0369f7299dd -u -S]
Nov 26 10:59:06:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 
4:0,virtio-blk,/virtual/openbsd/disk0.img -s 
5:0,virtio-net,tap0,mac=58:9c:fc:09:93:65 -s 6:0,passthru,4/0/0]
Nov 26 10:59:06:  [bhyve console: -l com1,/dev/nmdm-openbsd.1A]
Nov 26 10:59:06:  [bhyve iso device: -s 
3:0,ahci-cd,/virtual/.iso/install73.iso,ro]
Nov 26 10:59:06: starting bhyve (run 3)
Nov 26 11:01:46: bhyve exited with status 0
Nov 26 11:01:46: restarting
Nov 26 11:01:47: create file /virtual/openbsd/device.map
Nov 26 11:01:47:  -> (hd0) /virtual/openbsd/disk0.img
Nov 26 11:01:47: create file /virtual/openbsd/grub.cfg
Nov 26 11:01:47:  -> timeout=3
Nov 26 11:01:47:  -> menuentry 'openbsd (bhyve run)' {
Nov 26 11:01:47:  ->     root=hd0,1
Nov 26 11:01:47:  ->     kopenbsd -h com0 -r sd0a /bsd
Nov 26 11:01:47:  -> }
Nov 26 11:01:47: /usr/local/sbin/grub-bhyve -c /dev/nmdm-openbsd.1A -S 
-m /virtual/openbsd/device.map -M 768M -r host -d /virtual/openbsd openbsd
Nov 26 11:03:35:  [bhyve options: -c 1 -m 768M -AHP -w -U 
af22d4e4-2a87-11eb-b8fd-a0369f7299dd -u -S]
Nov 26 11:03:35:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 
4:0,virtio-blk,/virtual/openbsd/disk0.img -s 
5:0,virtio-net,tap0,mac=58:9c:fc:09:93:65 -s 6:0,passthru,4/0/0]
Nov 26 11:03:35:  [bhyve console: -l com1,/dev/nmdm-openbsd.1A]
Nov 26 11:03:35:  [bhyve iso device: -s 
3:0,ahci-cd,/virtual/.iso/install73.iso,ro]
Nov 26 11:03:35: starting bhyve (run 4)
Nov 26 11:04:16: bhyve exited with status 1
Nov 26 11:04:16: destroying network device tap0
Nov 26 11:04:16: stopped
Nov 26 11:07:32: initialising
Nov 26 11:07:32:  [loader: grub]
Nov 26 11:07:32:  [cpu: 1]
Nov 26 11:07:32:  [memory: 768M]
Nov 26 11:07:32:  [hostbridge: standard]
Nov 26 11:07:32:  [com ports: com1]
Nov 26 11:07:32:  [uuid: af22d4e4-2a87-11eb-b8fd-a0369f7299dd]
Nov 26 11:07:32:  [debug mode: no]
Nov 26 11:07:32:  [primary disk: disk0.img]
Nov 26 11:07:32:  [primary disk dev: file]
Nov 26 11:07:32: initialising network device tap0
Nov 26 11:07:32: booting
Nov 26 11:07:32: create file /virtual/openbsd/device.map
Nov 26 11:07:32:  -> (cd0) /virtual/.iso/openbsd74.iso
Nov 26 11:07:32:  -> (hd0) /virtual/openbsd/disk0.img
Nov 26 11:07:32: create file /virtual/openbsd/grub.cfg
Nov 26 11:07:32:  -> timeout=3
Nov 26 11:07:32:  -> menuentry 'openbsd (bhyve install)' {
Nov 26 11:07:32:  ->     root=cd0
Nov 26 11:07:32:  ->     kopenbsd -h com0 /7.4/amd64/bsd.rd
Nov 26 11:07:32:  -> }
Nov 26 11:07:32: /usr/local/sbin/grub-bhyve -c /dev/nmdm-openbsd.1A -S 
-m /virtual/openbsd/device.map -M 768M -r host -d /virtual/openbsd openbsd
Nov 26 11:07:35:  [bhyve options: -c 1 -m 768M -AHP -w -U 
af22d4e4-2a87-11eb-b8fd-a0369f7299dd -u -S]
Nov 26 11:07:35:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 
4:0,virtio-blk,/virtual/openbsd/disk0.img -s 
5:0,virtio-net,tap0,mac=58:9c:fc:09:93:65 -s 6:0,passthru,4/0/0]
Nov 26 11:07:35:  [bhyve console: -l com1,/dev/nmdm-openbsd.1A]
Nov 26 11:07:35:  [bhyve iso device: -s 
3:0,ahci-cd,/virtual/.iso/openbsd74.iso,ro]
Nov 26 11:07:35: starting bhyve (run 1)
Nov 26 11:07:40: bhyve exited with status 0
Nov 26 11:07:40: restarting
Nov 26 11:07:41: create file /virtual/openbsd/device.map
Nov 26 11:07:41:  -> (hd0) /virtual/openbsd/disk0.img
Nov 26 11:07:41: create file /virtual/openbsd/grub.cfg
Nov 26 11:07:41:  -> timeout=3
Nov 26 11:07:41:  -> menuentry 'openbsd (bhyve run)' {
Nov 26 11:07:41:  ->     root=hd0,1
Nov 26 11:07:41:  ->     kopenbsd -h com0 -r sd0a /bsd
Nov 26 11:07:41:  -> }
Nov 26 11:07:41: /usr/local/sbin/grub-bhyve -c /dev/nmdm-openbsd.1A -S 
-m /virtual/openbsd/device.map -M 768M -r host -d /virtual/openbsd openbsd
Nov 26 11:07:44:  [bhyve options: -c 1 -m 768M -AHP -w -U 
af22d4e4-2a87-11eb-b8fd-a0369f7299dd -u -S]
Nov 26 11:07:44:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 
4:0,virtio-blk,/virtual/openbsd/disk0.img -s 
5:0,virtio-net,tap0,mac=58:9c:fc:09:93:65 -s 6:0,passthru,4/0/0]
Nov 26 11:07:44:  [bhyve console: -l com1,/dev/nmdm-openbsd.1A]
Nov 26 11:07:44:  [bhyve iso device: -s 
3:0,ahci-cd,/virtual/.iso/openbsd74.iso,ro]
Nov 26 11:07:44: starting bhyve (run 2)
Nov 26 11:09:13: bhyve exited with status 1
Nov 26 11:09:13: destroying network device tap0
Nov 26 11:09:13: stopped
Nov 26 11:10:02: initialising
Nov 26 11:10:02:  [loader: grub]
Nov 26 11:10:02:  [cpu: 1]
Nov 26 11:10:02:  [memory: 768M]
Nov 26 11:10:02:  [hostbridge: standard]
Nov 26 11:10:02:  [com ports: com1]
Nov 26 11:10:02:  [uuid: af22d4e4-2a87-11eb-b8fd-a0369f7299dd]
Nov 26 11:10:02:  [debug mode: no]
Nov 26 11:10:02:  [primary disk: disk0.img]
Nov 26 11:10:02:  [primary disk dev: file]
Nov 26 11:10:02: initialising network device tap0
Nov 26 11:10:02: booting
Nov 26 11:10:02: create file /virtual/openbsd/device.map
Nov 26 11:10:02:  -> (cd0) /virtual/.iso/openbsd74.iso
Nov 26 11:10:02:  -> (hd0) /virtual/openbsd/disk0.img
Nov 26 11:10:02: create file /virtual/openbsd/grub.cfg
Nov 26 11:10:02:  -> timeout=3
Nov 26 11:10:02:  -> menuentry 'openbsd (bhyve install)' {
Nov 26 11:10:02:  ->     root=cd0
Nov 26 11:10:02:  ->     kopenbsd -h com0 /7.4/amd64/bsd.rd
Nov 26 11:10:02:  -> }
Nov 26 11:10:02: /usr/local/sbin/grub-bhyve -c /dev/nmdm-openbsd.1A -S 
-m /virtual/openbsd/device.map -M 768M -r host -d /virtual/openbsd openbsd
Nov 26 11:10:06:  [bhyve options: -c 1 -m 768M -AHP -U 
af22d4e4-2a87-11eb-b8fd-a0369f7299dd -u -S]
Nov 26 11:10:06:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 
4:0,virtio-blk,/virtual/openbsd/disk0.img -s 
5:0,virtio-net,tap0,mac=58:9c:fc:09:93:65 -s 6:0,passthru,4/0/0]
Nov 26 11:10:06:  [bhyve console: -l com1,/dev/nmdm-openbsd.1A]
Nov 26 11:10:06:  [bhyve iso device: -s 
3:0,ahci-cd,/virtual/.iso/openbsd74.iso,ro]
Nov 26 11:10:06: starting bhyve (run 1)
Nov 26 11:10:10: bhyve exited with status 0
Nov 26 11:10:10: restarting
Nov 26 11:10:11: create file /virtual/openbsd/device.map
Nov 26 11:10:11:  -> (hd0) /virtual/openbsd/disk0.img
Nov 26 11:10:11: create file /virtual/openbsd/grub.cfg
Nov 26 11:10:11:  -> timeout=3
Nov 26 11:10:11:  -> menuentry 'openbsd (bhyve run)' {
Nov 26 11:10:11:  ->     root=hd0,1
Nov 26 11:10:11:  ->     kopenbsd -h com0 -r sd0a /bsd
Nov 26 11:10:11:  -> }
Nov 26 11:10:11: /usr/local/sbin/grub-bhyve -c /dev/nmdm-openbsd.1A -S 
-m /virtual/openbsd/device.map -M 768M -r host -d /virtual/openbsd openbsd
Nov 26 11:10:15:  [bhyve options: -c 1 -m 768M -AHP -U 
af22d4e4-2a87-11eb-b8fd-a0369f7299dd -u -S]
Nov 26 11:10:15:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 
4:0,virtio-blk,/virtual/openbsd/disk0.img -s 
5:0,virtio-net,tap0,mac=58:9c:fc:09:93:65 -s 6:0,passthru,4/0/0]
Nov 26 11:10:15:  [bhyve console: -l com1,/dev/nmdm-openbsd.1A]
Nov 26 11:10:15:  [bhyve iso device: -s 
3:0,ahci-cd,/virtual/.iso/openbsd74.iso,ro]
Nov 26 11:10:15: starting bhyve (run 2)
Nov 26 11:10:52: bhyve exited with status 1
Nov 26 11:10:52: destroying network device tap0
Nov 26 11:10:52: stopped


-- 
Public Keys:
PGP KeyID = 0x5F22FDC1
GnuPG KeyID = 0x620836CF