From nobody Sat Oct 26 13:28:11 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XbL7k00fnz5ZlHS; Sat, 26 Oct 2024 13:28:18 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XbL7j1Wndz4Kjm; Sat, 26 Oct 2024 13:28:17 +0000 (UTC) (envelope-from markjdb@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-io1-xd32.google.com with SMTP id ca18e2360f4ac-83ac4dacaf9so110812539f.2; Sat, 26 Oct 2024 06:28:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729949295; x=1730554095; darn=freebsd.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:from:to:cc:subject:date:message-id :reply-to; bh=2JUrFFoEnFcKu37sIlpTjfbtmFnVb4qMPVMy4toFWsE=; b=I1yT1Ega0gCMUcmdovJuhIdn3IIMJX0WTs4SJIZnGjV/BC5aTJszS+UfOvuAtQKLAE re4PAAhEKE2pKnLmF75/EtxaYLUs+jW/qL7njIqpXrLaYgZ2y4fDk2xKTg6YDfqy8xsr YZioBr1rfrDDgLTrAv4lfkDhpDoaGWnRCk3bad/4enFvOqM6adIokpo7u+VjOeS19Tda zWLKQxNnmqpD73pyMz0bZkI6SygFpj0i3YdpxqqPl02TnvKYanqQftItKMNTKdIbQW18 SViczmI1Y/t36AqbuwQX60hMI7oMvLpjScFohhk+dMfogso2h2mrFmPzN3bPh/Fz/ZaF YhWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729949295; x=1730554095; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2JUrFFoEnFcKu37sIlpTjfbtmFnVb4qMPVMy4toFWsE=; b=ZfObcFCZonE4Z6x+p0p8SOfh6gKIbxARraVUqMtAYtwh85oTzJhRF0QhOePcGiE3i7 goMDHqI9iMRAE6Q2eH734B8ndqPqDJda3KAuUD/SEGzpmCIPTxOXcaY3P4TXVjA4crxk nmdOgpY00tPNFDuf9rzaEphZIu4kPnBge09iHvBphx++zR6ewcYTujjSgU55GK3Rn5zZ uTwghbCfCyrXer26GsK29TRCxg1pvGru81HpqeeNXg9pgMBOI/ihCSwbTxEnieHe0h4z kvQ0pv8BMprFsY6ggaAwBn7i7SvDzZNwnuIBo8AbJggDcBoZc1VQaF+nu74XEMhSRaas Hn/w== X-Forwarded-Encrypted: i=1; AJvYcCXX9AuPMohk6FdXBO+w0Q3ME4TBMBw3uygQBSlntPtNDTbarH5KHTicaAcUyDPinuPGZfgIiR9p5/OreK/8EA==@freebsd.org X-Gm-Message-State: AOJu0YyiTYFHc+W+H+Pj4SKXUzr00p6zsUffvMK+MKkRNWAkB+KdVjAB zZwhnblEXefSBvA21ytMCqydxWqw9jx8VP1jHSlgsjpt2R8wBkVoG4/Ugw== X-Google-Smtp-Source: AGHT+IGmIYvBTOS6HO15j1JeaWxWEnyIDg9jGg9ZZDb4XOnMba6KbK4uFwmGbw/Dr6eHDijp1+UaPw== X-Received: by 2002:a05:6602:2b0c:b0:82c:eeaa:b1e0 with SMTP id ca18e2360f4ac-83b1c5bebf8mr256683839f.11.1729949295340; Sat, 26 Oct 2024 06:28:15 -0700 (PDT) Received: from nuc (192-0-220-237.cpe.teksavvy.com. [192.0.220.237]) by smtp.gmail.com with ESMTPSA id 8926c6da1cb9f-4dc7260f159sm733393173.37.2024.10.26.06.28.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 26 Oct 2024 06:28:14 -0700 (PDT) Date: Sat, 26 Oct 2024 09:28:11 -0400 From: Mark Johnston To: Guido Falsi Cc: Current FreeBSD , "virtualization@freebsd.org" Subject: Re: bhyve regression (head): windows VMs failing with error 0xc000021a Message-ID: References: <6a2a37b3-353e-40ca-b8a9-f4ef97733da8@madpilot.net> <48eb1621-4b22-4080-8e80-059d475244f3@madpilot.net> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <48eb1621-4b22-4080-8e80-059d475244f3@madpilot.net> X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4XbL7j1Wndz4Kjm X-Spamd-Bar: ---- On Fri, Oct 25, 2024 at 11:12:48PM +0200, Guido Falsi wrote: > On 25/10/24 22:49, Mark Johnston wrote: > > On Fri, Oct 25, 2024 at 09:24:13PM +0200, Guido Falsi wrote: > > > Hi, > > > > > > I've recently updated my current machines to git commit > > > 525a177c165740fc697df3de5b92e58b3b41477c (Sun Oct 20 22:43:41 2024 -0800) > > > and just have Windows 10 VMs fail to start in bhyve with the error in the > > > subject. > > > > > > I've been unable to recover them with usual tricks (automatic recovery, > > > chkdsk, and other tools provided by the OS). Looks like the machine fails to > > > read C: after boot. > > > > > > These machines were working fine before the update, so my suspect is that > > > some recent change in bhyve is causing the issue and the VMs would be > > > otherwise fine. > > > > > > The VMs have their filesystems in compressed zvols. > > > > > > Anyone has an idea or can point to some change I can test reverting? > > > > Just a guess, but you might try adding "-o pci.enable_bars=true" to the > > bhyve command line arguments > > Tried but it looks like it made no difference. > > > > > > I an also try bisecting, I'd guess the issue is quite recent. > > > > Which revision did you update from? > > I updated from git commit 450a6690f557493bd33d8f3666b22ddc5150703b (Thu Sep > 19 11:49:40 2024 -0500) > > In the while I noticed some commits to TPM emulation/passthorugh, maybe > they're related? It might be, but I don't see any TPM devices configured in the invocation below. There was a number of changes to usr.sbin/bhyve in that window, so reverting them one by one would probably turn up the culprit. Or, you need to pick up commit 8c8ebbb045185396083cd3e4d333fe1851930ee7, given that you're using AHCI emulation. > > > > > Thanks in advance for any advice. > > > > > > Obviously if any further info is needed just ask. > > > > What command line arguments are passed to bhyve? What boot firmware are > > you using? > > I'm using vm-bhyve, from its log the options should have been: > > -c 4,sockets=1,cores=2,threads=2 -m 4G -AHPw -l > bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -o pci.enable_bars=true > -U xxx -s 0,hostbridge -s 31,lpc -s > 4:0,ahci,hd:/dev/zvol/zroot/bhyve/W64/disk0 -s 5:0,virtio-net,tap1,mac=xxx > -s 6:0,fbuf,tcp=127.0.0.1:5900,w=1600,h=900 -s 7:0,xhci,tablet -s > 8:0,hda,play=/dev/dsp1 -l com1,stdio > > (replaced MAC and UUID with xxx) > > for firmware I'm using bhyve-firmware-1.0_2 from sysutils/bhyve-firmware