From nobody Thu Sep 21 06:36:35 2023 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Rrlzv2ncsz4td6s for ; Thu, 21 Sep 2023 06:36:43 +0000 (UTC) (envelope-from dmarquess@gmail.com) Received: from mail-oo1-xc2d.google.com (mail-oo1-xc2d.google.com [IPv6:2607:f8b0:4864:20::c2d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Rrlzv0f31z3ZKZ; Thu, 21 Sep 2023 06:36:43 +0000 (UTC) (envelope-from dmarquess@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oo1-xc2d.google.com with SMTP id 006d021491bc7-5733aa10291so383869eaf.3; Wed, 20 Sep 2023 23:36:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1695278202; x=1695883002; darn=freebsd.org; h=mime-version:subject:references:in-reply-to:message-id:to:from:date :from:to:cc:subject:date:message-id:reply-to; bh=DlS9200UAenZSYzcESzizrHvu0zzZuMbW32S7m1WGHo=; b=Dc62qByORD+FJJkUPBKijoTMapmO7rilvn/vS2BHTV0/U4cqD/L/i8YsG2XJydRh36 nodfNivybk1qvq8uIjHk8PkKXY0wOUZFsrVATD+uXUQDUvt/LvS5OKz41hrExuxwpPay Q1TAj8jBTyEu+IuRKs2DYCTOcs+6q7u1VLN1Pcu2iVGsL7vhPguvFJHDlPxqU+EfiNR1 UjNY+HIBaQK9N+3guyovxPSg1POn2YTyUqPgzycluNIQJ7sCUi43GSSMMkJUn3pubbEV LFs/v1Z647Zz1M72QwSLMR/fBgqqbBrgjU4tgFuzC3eVthdCo7hqyxMeSb1NU4aXkQIg EYBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695278202; x=1695883002; h=mime-version:subject:references:in-reply-to:message-id:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=DlS9200UAenZSYzcESzizrHvu0zzZuMbW32S7m1WGHo=; b=gEu5RKq56VPfLt7qSksLM/nfgH9ZkGlljEtybtK95Djp4MrVZPkKGwe1Um1U779cRl upWYVH5OKZX4VpwcBdUuIPZfeVH5Lqy1uo5kRpPjeMjRNsZeNrEwsDrJ/umNWe/uVaO1 4UlH4+l4WuVC4OKrZe05hF+W3KQCk7rM2YefyPGJQpNKxKZQzfpnfZ/gpzDDMAsfjTwX 0PzDBO43vdTFD1J/8Y/9ja3V4s6gzPilD3APk8C1/U3mfis4SGpTPz56gK8MB2N/JavP Rqi05CgmqBdoxmOQmOkBVWL346SB0vinAYwaDmUCTpFBMIWGc5QR1yQrWIkTocLpfF3U Hsvg== X-Gm-Message-State: AOJu0YyeH41LAwLJVZyydQxqNQElMSfbchBl6iL2ulXGa4SgBoCudChn 7FbFUaPLTnJ+c354XZGsfMXT7jGKvMqZWg== X-Google-Smtp-Source: AGHT+IHP+Q2S15a0HC1jOeXvDNHL7cFzkoOrK4yDQAUKcEtZJRH/QwoR6ACas2h2C9YmJmjzPxZ56g== X-Received: by 2002:a4a:7611:0:b0:571:2b59:d697 with SMTP id t17-20020a4a7611000000b005712b59d697mr4974005ooc.0.1695278201821; Wed, 20 Sep 2023 23:36:41 -0700 (PDT) Received: from [2001:470:bc52:2:402a:640c:70:0] ([2001:470:bc52:2:860:15c0:3694:e826]) by smtp.gmail.com with ESMTPSA id t25-20020a9d7759000000b006b8e8884f2fsm339420otl.51.2023.09.20.23.36.41 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 20 Sep 2023 23:36:41 -0700 (PDT) Date: Thu, 21 Sep 2023 01:36:35 -0500 From: Dustin Marquess To: FreeBSD virtualization , =?utf-8?Q?Corvin_K=C3=B6hne?= Message-ID: <0ab66830-c168-4738-b1f2-d1071acebbe3@Spark> In-Reply-To: <3cf178349ce0483b031bc5705be33a177dabf1ba.camel@FreeBSD.org> References: <3c3ccffc-5a57-4472-8d16-5b5cdf5da862@Spark> <3cf178349ce0483b031bc5705be33a177dabf1ba.camel@FreeBSD.org> Subject: Re: bhyve issues in 15-CURRENT X-Readdle-Message-ID: 0ab66830-c168-4738-b1f2-d1071acebbe3@Spark List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="650be478_43143d5b_2bc8" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4Rrlzv0f31z3ZKZ --650be478_43143d5b_2bc8 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Sep 21, 2023 at 1:29=E2=80=AFAM -0500, Corvin K=C3=B6hne , wrote: > On Wed, 2023-09-20 at 19:02 -0500, Dustin Marquess wrote: > > > > > > > > I just upgraded from a kernel+userland of 15-CURRENT from a release > > 20 days ago (git commit 565c887) to one today (git commit e39e6be). > > Afterwards, I've noticed two things that are probably really the same= > > thing: > > > > - My Windows Server 2023 VM says that there's not enough resources > > for COM2 & COM4 > > - My OpenBSD 7.3 VM dies with: > > > > acpicmos0 at acpi0 > > =22Bhyve=5FV=5FGen=5FCounter=5FV1=22 at acpi0 not configured > > cpu0: using VERW MDS workaround > > pvbus0 at mainbus0: bhyve > > pci0 at mainbus0 bus 0 > > 0:3:0: io address conflict 0xc000/0x80 > > 0:5:0: io address conflict 0xc080/0x40 > > pchb0 at pci0 dev 0 function 0 vendor =22AMD=22, unknown product 0x74= 32 > > rev 0x00 > > virtio0 at pci0 dev 3 function 0 =22Qumranet Virtio Storage=22 rev > > 0x00virtio0: can't map i/o space > > : Cannot attach (5) > > virtio1 at pci0 dev 5 function 0 =22Qumranet Virtio Network=22 rev 0x= 00 > > vio0 at virtio1: address 1e:17:37:23:2f:cb > > virtio1: msix per-VQ > > > > In this case, 0:3:0 is the virtio-blk device and 0:5:0 is the virtio-= > > net device. > > > > If I boot it using a snapshot install74.img then it dies at: > > > > acpicmos0 at acpi0 > > =22Bhyve=5FV=5FGen=5FCounter=5FV1=22 at acpi0 not configured > > cpu0: using VERW MDS workaround > > pvbus0 at mainbus0: bhyve > > pci0 at mainbus0 bus 0 > > 0:2:0: io address conflict 0xc080/0x80 > > 0:3:0: io address conflict 0xc000/0x80 > > 0:5:0: io address conflict 0xc100/0x40 > > pchb0 at pci0 dev 0 function 0 vendor =22AMD=22, unknown product 0x74= 32 > > rev 0x00 > > virtio0 at pci0 dev 2 function 0 =22Qumranet Virtio Storage=22 rev > > 0x00virtio0: can't map i/o space > > : Cannot attach (5) > > virtio1 at pci0 dev 3 function 0 =22Qumranet Virtio Storage=22 rev 0x= 00 > > vioblk0 at virtio1 > > scsibus0 at vioblk0: 1 targets > > sd0 at scsibus0 targ 0 lun 0: > > sd0: 8192MB, 512 bytes/sector, 16777216 sectors > > virtio1: msix per-VQ > > virtio2 at pci0 dev 5 function 0 =22Qumranet Virtio Network=22 rev 0x= 00 > > vio0 at virtio2: address ff:ff:ff:ff:ff:ff > > panic: vq=5Fsize not power of two: 65535 > > > > 0:3:0 are virtio-blk and 0:5:0 is virtio-net, as above. 0:2:0 is the > > extra virtio-blk for the .img file. > > > > Any ideas on what change caused this=3F > Please make sure to call bhyve with the -A option. Yep, my script runs it as: bhyve -A -D -H -P -w -S -u -c =24CPUS -m =24RAM =5C -s 0,amd=5Fhostbridge =5C -s 3,virtio-blk,/dev/zvol/tank/vm/=24VM =5C -s 5,virtio-net,tap0,mac=3D1e:17:37:23:2f:cb =5C -s 31,lpc =5C -l com1,stdio -l bootrom,/usr/local/share/uefi-firmware/BHYVE=5FUE=46I.fd= =24VM So -A is there already. -Dustin --650be478_43143d5b_2bc8 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline
On Sep 21, 2023 at 1:29=E2=80=AFAM -0500, Corvin K=C3= =B6hne <corvink=40freebsd.org>, wrote:
On Wed, 2023-09-20 at 19:02 -0500, Dustin Marquess wrote:



I just upgraded from a kernel+userland of 15-CURRENT from a release
= 20 days ago (git commit 565c887) to one today (git commit e39e6be).
= Afterwards, I've noticed two things that are probably really the same
thing:

- My Windows Server 2023 VM says that there's not enough resources
for COM2 & COM4
- My OpenBSD 7.3 VM dies with:

acpicmos0 at acpi0
=22Bhyve=5FV=5FGen=5FCounter=5FV1=22 at acpi0 not configured
cpu0: using VERW MDS workaround
pvbus0 at mainbus0: bhyve
pci0 at mainbus0 bus 0
0:3:0: io address conflict 0xc000/0x80
0:5:0: io address conflict 0xc080/0x40
pchb0 at pci0 dev 0 function 0 vendor =22AMD=22, unknown product 0x7432 rev 0x00
virtio0 at pci0 dev 3 function 0 =22Qumranet Virtio Storage=22 rev
0x00virtio0: can't map i/o space
: Cannot attach (5)
virtio1 at pci0 dev 5 function 0 =22Qumranet Virtio Network=22 rev 0x00 vio0 at virtio1: address 1e:17:37:23:2f:cb
virtio1: msix per-VQ

In this case, 0:3:0 is the virtio-blk device and 0:5:0 is the virtio-
net device.

If I boot it using a snapshot install74.img then it dies at:

acpicmos0 at acpi0
=22Bhyve=5FV=5FGen=5FCounter=5FV1=22 at acpi0 not configured
cpu0: using VERW MDS workaround
pvbus0 at mainbus0: bhyve
pci0 at mainbus0 bus 0
0:2:0: io address conflict 0xc080/0x80
0:3:0: io address conflict 0xc000/0x80
0:5:0: io address conflict 0xc100/0x40
pchb0 at pci0 dev 0 function 0 vendor =22AMD=22, unknown product 0x7432 rev 0x00
virtio0 at pci0 dev 2 function 0 =22Qumranet Virtio Storage=22 rev
0x00virtio0: can't map i/o space
: Cannot attach (5)
virtio1 at pci0 dev 3 function 0 =22Qumranet Virtio Storage=22 rev 0x00 vioblk0 at virtio1
scsibus0 at vioblk0: 1 targets
sd0 at scsibus0 targ 0 lun 0: <VirtIO, Block Device, >
sd0: 8192MB, 512 bytes/sector, 16777216 sectors
virtio1: msix per-VQ
virtio2 at pci0 dev 5 function 0 =22Qumranet Virtio Network=22 rev 0x00 vio0 at virtio2: address ff:ff:ff:ff:ff:ff
panic: vq=5Fsize not power of two: 65535

0:3:0 are virtio-blk and 0:5:0 is virtio-net, as above. 0:2:0 is the
extra virtio-blk for the .img file.

Any ideas on what change caused this=3F
Please make sure to call bhyve with the -A option.

Yep, my script runs it as:

bhyve -A -D -H -P -w -S -u -c =24CPUS -m =24RAM =5C
-s 0,amd=5Fhostbridge =5C
-s 3,virtio-blk,/dev/zvol/tank/vm/=24VM =5C
-s 5,virtio-net,tap0,mac=3D1e:17:37:23:2f:cb =5C
-s 31,lpc =5C
-l com1,stdio -l bootrom,/usr/local/share/uefi-firmware/BHYVE=5FUE=46I.fd= =24VM

So -A is there already.

-Dustin
--650be478_43143d5b_2bc8--