From nobody Fri Oct 25 21:12:48 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XZwVF2K4jz5b93K; Fri, 25 Oct 2024 21:12:53 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from vogon.madpilot.net (vogon.madpilot.net [159.69.1.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XZwVF09Zsz4tt7; Fri, 25 Oct 2024 21:12:53 +0000 (UTC) (envelope-from mad@madpilot.net) Authentication-Results: mx1.freebsd.org; none Received: from mail (mail [IPv6:fd5c:5351:d272::3]) by vogon.madpilot.net (Postfix) with ESMTP id 4XZwVC1gqlzL80M; Fri, 25 Oct 2024 23:12:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=madpilot.net; h= content-transfer-encoding:content-type:content-type:in-reply-to :from:from:content-language:references:subject:subject:date:date :message-id:received; s=cyq4qetkgngm; t=1729890769; x= 1731705170; bh=z1ugqF2nZIxiUcPP92Tjz13fpoRfMN28AeI6w5YHf/w=; b=H 6tFF4SbkA5oAEtvjexBax5XXuMY5CqLOEfTrSmF4xnu7L+eKso1KFsSSVumif049 UulyNqi2gNcs8HWum6GWvS0nlvJ8uFfdB5H4ZMbGIjp7VUPD16hpMNxWmKx9O6Xe AYhgVWQqNz7Z2un2f3vLVpiZhV3hNgooEYtkJaajuBJjgMJGHwS8MEIP6L0E8/4D F6P3xjv+hm1MuNbKUV6/ex0e+WimYWnTqIj399FmWfw/F3Fi9lWGOt4HUHuO5bQg f7+zRrfFvywa13uLscl1amOwSFc31kQ+reZiwXZOGPrKabHU+LajnXnBd+5+2P9v iUpxk5hby3M7zbvANzo1w== Received: from vogon.madpilot.net ([IPv6:fd5c:5351:d272::3]) by mail (vogon.madpilot.net [IPv6:fd5c:5351:d272::3]) (amavisd-new, port 10026) with ESMTP id ii5u5nt3QSRB; Fri, 25 Oct 2024 23:12:49 +0200 (CEST) Message-ID: <48eb1621-4b22-4080-8e80-059d475244f3@madpilot.net> Date: Fri, 25 Oct 2024 23:12:48 +0200 Subject: Re: bhyve regression (head): windows VMs failing with error 0xc000021a To: Mark Johnston Cc: Current FreeBSD , "virtualization@freebsd.org" References: <6a2a37b3-353e-40ca-b8a9-f4ef97733da8@madpilot.net> Content-Language: en-US From: Guido Falsi Autocrypt: addr=mad@madpilot.net; keydata= xsBNBE+G+l0BCADi/WBQ0aRJfnE7LBPsM0G3m/m3Yx7OPu4iYFvS84xawmRHtCNjWIntsxuX fptkmEo3Rsw816WUrek8dxoUAYdHd+EcpBcnnDzfDH5LW/TZ4gbrFezrHPdRp7wdxi23GN80 qPwHEwXuF0X4Wy5V0OO8B6VT/nA0ADYnBDhXS52HGIJ/GCUjgqJn+phDTdCFLvrSFdmgx4Wl c0W5Z1p5cmDF9l8L/hc959AeyNf7I9dXnjekGM9gVv7UDUYzCifR3U8T0fnfdMmS8NeI9NC+ wuREpRO4lKOkTnj9TtQJRiptlhcHQiAlG1cFqs7EQo57Tqq6cxD1FycZJLuC32bGbgalABEB AAHNHkd1aWRvIEZhbHNpIDxtYWRAbWFkcGlsb3QubmV0PsLAeQQTAQgAIwIbAwIeAQIXgAUL CQgHAwUVCgkICwQWAgMBBQJS79AgAhkBAAoJEBrmhg5Wy9KTc0kH/RO64ORBlTbTHaUaOj8F Je5O5NU2Pt9Cyt5ZWBRvxntr1zPTJGKRPS9ihlIfqT4ZvEngQGp57EUyFbCpI0UWasTerImM tt5WACnGmCzUTB39UXx8Oy4b1EgWeTJQ747e/F1mQLXTNa6ijRBE9fYlTb4gAkPN88/wVV9v 3PZozKLTg16ghBzHM/P7Lk8L7clPEZChX1FTa/6eSt3nvzfCuTMZbBPJF/ph+q1KyPqRgVfh tyhu5dvgMoPz/ni41IfeSrkJTD5RXzdyGR9q4Z1NYeBsLkRjC4LxKAP5KqUsvlOUjKvO1byj ApYdMarol+IGkaSk9e3zVYAJkWKjn/ni8XbOwU0EUxB7QQEQAKFhrDceoPdK/IHDSmoj6SQY isvM7VdhcleS7E9DoEAVt7yMbf6HbbMVTTY6ckvwTWQssywLBXNVqxgc4WLJjzfUhgef+WE7 5M3+WFYlOVQLGZY/zEVgma1raYnOHNAOzeHLDmEXjbZP6vGAeDyBbGfQPpE7qGYZ7ubeT3Xw QO+PklcCrvOPj2ZPcAxGNS2xVU/LzONqCrJqLMJSIcCdsbiSP4G5PnDFHtMokaTY6OEr8OEQ fOAerhcHUa/z7Uu8YtmaqKH+QGkE/WEgaRqSiTnv0JOTD+DxehaqvoKPPZ++2NpCZMHB2i6A /xifmQwEiIjEXtcueBRzkNUQkxhqZyS13SrhocL9ydtaVPBzZatAEjUDDEJmAMLVFs45qfyh MiNapHJo2n3MW/E5omqCvEkDdWX/en3P7CK2TemeaDghMsgkNKax/z0wNo5UZCkOPOz0xpNi UilOVbkuezZZNg65741qee2lfXhQIaZ66yT7hphc/N/z3PIAtLeze4u1VR2EXAuZ2sWAdlKC NTlJMsaU/x70BV11Wd/ypnVzM68dfdQIIAj1iMFAD/lXGlEUmKXg5Ov2VQDlTntQoanCYrAg +8CttPzjrydgLZFq3hrtQmfc0se5yv1WHS69+BsUOG09RvvawUDZxUjW19kyeN9THaNRgow3 kSuArUp6zSmJABEBAAHCwF8EGAEIAAkFAlMQe0ECGwwACgkQGuaGDlbL0pMN5wgA4bCkX/qw EVC06ToeR6C2putmSWQMgpDaqrv65Hubo+QGmg2P4ewTYQQ4g6oYWS03qHxqVVWhKz7FjfrV +dH8qbCLfSgIcvdBha7ayGZVrsiuMLKGbw36fcmkZPpSDOfHcP0XH8Z+u9CWj0xUkTxAlZ/7 i6gYSUpG2JWNtdmE/X8VVEyXusCLwy0K0BI60A/4dRTIX3C4QKrJ3ZbUXegz70ynjHf+lQMZ 9IZKASoRMuS5FozPQh6abvmwZEPdf5I9riUElzvHrqJ8Bx0t3Pujdoth+yNHpnBxrtO8LkQd rQ58P0SwcaIX33T2U9pG8bhu5YVR88FQ8OQ0cEsPBpDncg== In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:159.69.0.0/16, country:DE] X-Rspamd-Queue-Id: 4XZwVF09Zsz4tt7 X-Spamd-Bar: ---- List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org On 25/10/24 22:49, Mark Johnston wrote: > On Fri, Oct 25, 2024 at 09:24:13PM +0200, Guido Falsi wrote: >> Hi, >> >> I've recently updated my current machines to git commit >> 525a177c165740fc697df3de5b92e58b3b41477c (Sun Oct 20 22:43:41 2024 -0800) >> and just have Windows 10 VMs fail to start in bhyve with the error in the >> subject. >> >> I've been unable to recover them with usual tricks (automatic recovery, >> chkdsk, and other tools provided by the OS). Looks like the machine fails to >> read C: after boot. >> >> These machines were working fine before the update, so my suspect is that >> some recent change in bhyve is causing the issue and the VMs would be >> otherwise fine. >> >> The VMs have their filesystems in compressed zvols. >> >> Anyone has an idea or can point to some change I can test reverting? > > Just a guess, but you might try adding "-o pci.enable_bars=true" to the > bhyve command line arguments Tried but it looks like it made no difference. > >> I an also try bisecting, I'd guess the issue is quite recent. > > Which revision did you update from? I updated from git commit 450a6690f557493bd33d8f3666b22ddc5150703b (Thu Sep 19 11:49:40 2024 -0500) In the while I noticed some commits to TPM emulation/passthorugh, maybe they're related? > >> Thanks in advance for any advice. >> >> Obviously if any further info is needed just ask. > > What command line arguments are passed to bhyve? What boot firmware are > you using? I'm using vm-bhyve, from its log the options should have been: -c 4,sockets=1,cores=2,threads=2 -m 4G -AHPw -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -o pci.enable_bars=true -U xxx -s 0,hostbridge -s 31,lpc -s 4:0,ahci,hd:/dev/zvol/zroot/bhyve/W64/disk0 -s 5:0,virtio-net,tap1,mac=xxx -s 6:0,fbuf,tcp=127.0.0.1:5900,w=1600,h=900 -s 7:0,xhci,tablet -s 8:0,hda,play=/dev/dsp1 -l com1,stdio (replaced MAC and UUID with xxx) for firmware I'm using bhyve-firmware-1.0_2 from sysutils/bhyve-firmware -- Guido Falsi