From nobody Mon Aug 01 17:24:44 2022 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4LxQ3h4Bztz4X7gK; Mon, 1 Aug 2022 17:24:48 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com [IPv6:2a00:1450:4864:20::136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LxQ3g5f6wz3d2P; Mon, 1 Aug 2022 17:24:47 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: by mail-lf1-x136.google.com with SMTP id w15so18332009lft.11; Mon, 01 Aug 2022 10:24:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:from:to:cc; bh=m1c/OzKZuNkJ0EbEGrpQm7MHgNhmFqH8J++vfx/GsPM=; b=jgsG+7VohWKsmUtXoHK9hSBIUDooLKdXok2wo6yPgbc8G7+LW0ia6B9adM3OM4+lsZ 9oxa7tWmqhlV6ywzCsW5wFYBCLDdCOMPs0AKYnLL0BkneudnN4/jySyKuQVlUyuaaiyJ vMEixnujK6JDK84BmDgtvaTeg+nzqebs0+1ZPmQQRO/6I2Pul5yIr6Q5sM/Th+EaZCXz VMSo1SzvF4uJS1NDsGaRtbZs/IDCn4l98HGCkFXyG/oQAcSeik+uG4ZeUYGZxs0RUl63 Ox88CSZKHlSF5vi8yUPNwaqGGAYQVa7GwEJEIRbZkE1iuAbGhiVPPSVjRGmVSCdkLZOA Bmuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:x-gm-message-state:from:to:cc; bh=m1c/OzKZuNkJ0EbEGrpQm7MHgNhmFqH8J++vfx/GsPM=; b=1pX6i9zZhr0T8Ispj5ALpHCP+WD7uCFfdhhdTa4nMsUoqUvn+VPKtWRNcjuizG24kc 7oQznoOa3QJji3ab9/7T9b6PNxKybCbWUGk07TLvFdbtjuHa12gPPbqBH8kOaZ42J+RI UkJ4apU/FARAtKOZima1wG9SaYEPeZNpNhXnfAgG+L7rD7awVMWmk5hw4XEX+lnbwgQ/ +teRGOsL4f/sNYTt2ordpD2CNMr/w6uPAOjWKWpfuJ2/SU7XXxF6zwMwvp6hxiLhIzlp a5GzJdnM+JNr1R1z6r0vQ9l6Qt/xIvKByHwq0JrQRDE/nfs+z0nybJgo9CwyOKCU1HYX iPrg== X-Gm-Message-State: AJIora/qOSFE44KwZqhkSEQbleuXKwHSeJS41L95eRyq4xkfHJ6qFPFb Z4WtiQsOdh6QImWiAI1aNH3wIt0rWaaVZg== X-Google-Smtp-Source: AGRyM1t43HyNsfMGqjZk3Wfgc9ZF3olNw1wHyEdO3cEiG78CjvejKWy/zwp1dYb9d3Yw7ql48u4EJQ== X-Received: by 2002:a19:8c04:0:b0:47f:65b7:bf11 with SMTP id o4-20020a198c04000000b0047f65b7bf11mr6537348lfd.630.1659374685594; Mon, 01 Aug 2022 10:24:45 -0700 (PDT) Received: from smtpclient.apple ([188.187.60.230]) by smtp.gmail.com with ESMTPSA id o1-20020a056512052100b0048a98b7bad3sm931852lfc.197.2022.08.01.10.24.44 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 01 Aug 2022 10:24:44 -0700 (PDT) From: Vitaliy Gusev Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_FA2EEDB9-6782-4E0D-9AFD-58F553637AE0" List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.100.31\)) Subject: Re: problem with bhyve, ryzen 5800x, freebsd guest Date: Mon, 1 Aug 2022 20:24:44 +0300 In-Reply-To: <70b44a39-1df2-0f4e-42ac-9710ab82d028@FreeBSD.org> Cc: current@freebsd.org, virtualization@freebsd.org To: Andriy Gapon References: <70b44a39-1df2-0f4e-42ac-9710ab82d028@FreeBSD.org> X-Mailer: Apple Mail (2.3696.100.31) X-Rspamd-Queue-Id: 4LxQ3g5f6wz3d2P X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=jgsG+7Vo; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of gusev.vitaliy@gmail.com designates 2a00:1450:4864:20::136 as permitted sender) smtp.mailfrom=gusev.vitaliy@gmail.com X-Spamd-Result: default: False [-3.30 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.982]; NEURAL_HAM_MEDIUM(-0.82)[-0.818]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; MID_RHS_MATCH_FROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::136:from]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_SOME(0.00)[]; TAGGED_FROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; MLMMJ_DEST(0.00)[current@freebsd.org,virtualization@freebsd.org] X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_FA2EEDB9-6782-4E0D-9AFD-58F553637AE0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Interesting enough. It would be nice if you find out what exactly = triggered your VM to reset. =E2=80=94 Vitaliy > On 1 Aug 2022, at 17:39, Andriy Gapon wrote: >=20 > On 2022-07-10 20:28, Gleb Smirnoff wrote: >> On Thu, Jul 07, 2022 at 03:29:04PM +0300, Andriy Gapon wrote: >> A> I have a strange issue with running an 'appliance' image based on >> A> FreeBSD 12 in bhyve on a machine with Ryzen 5800x processor. >> A> >> A> The problem is that the guest would run for a while and then the = host >> A> would suddenly reset itself. It appears like a triple fault or >> A> something with similar consequences. >> A> >> A> The time may be from a few dozens of minutes to many hours. >> A> >> A> Just to be clear, no such thing occurs if I do not run the guest. >> A> Also, I have an older AMD system (pre-Zen), the problem does not = happen >> A> there. >> A> A vanilla FreeBSD 12.3 installation that just sits idle also does = not >> A> cause the problem. >> A> >> A> Does anyone have an idea what the problem could be? >> A> What workaround or diagnostics to try? >> A> Anybody else seen something like this? >> A> >> A> Since it's the host that resets it would be hard to capture any = traces. >> I also run bhyve on Ryzen since late 2021 and never had such an = issue. >> But not FreeBSD 12, I run the head. >=20 >=20 > Thank you everyone who responded. It seems that the problem was with = some BIOS configuration changes, probably related to the power settings. > Once I reset everything to factory defaults (plus some minimum "safe" = and well-understood changes) the problem went away. > It's really surprising that I saw it only with bhyve and only with the = particular kind of VMs. Perhaps there was a workload pattern that = triggered a hardware bug or overloaded some specific module. >=20 > Anyways, sorry for the noise and thank you for the help. >=20 > --=20 > Andriy Gapon >=20 >=20 > https://standforukraine.com > https://razomforukraine.org --Apple-Mail=_FA2EEDB9-6782-4E0D-9AFD-58F553637AE0 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Interesting enough. It would be nice if you find out what = exactly triggered your VM to reset.

=E2=80=94
Vitaliy

On 1 Aug 2022, at 17:39, Andriy Gapon <avg@FreeBSD.org> = wrote:

On 2022-07-10 20:28, Gleb Smirnoff wrote:
On = Thu, Jul 07, 2022 at 03:29:04PM +0300, Andriy Gapon wrote:
A> I have a strange issue with running an 'appliance' = image based on
A> FreeBSD 12 in bhyve on a machine with = Ryzen 5800x processor.
A>
A> The = problem is that the guest would run for a while and then the host
A> would suddenly reset itself. It appears like a triple = fault or
A> something with similar consequences.
A>
A> The time may be from a few dozens = of minutes to many hours.
A>
A> Just = to be clear, no such thing occurs if I do not run the guest.
A> Also, I have an older AMD system (pre-Zen), the problem = does not happen
A> there.
A> A vanilla = FreeBSD 12.3 installation that just sits idle also does not
A> cause the problem.
A>
A> Does anyone have an idea what the problem could be?
A> What workaround or diagnostics to try?
A> Anybody else seen something like this?
A>
A> Since it's the host that resets it = would be hard to capture any traces.
I also run bhyve on = Ryzen since late 2021 and never had such an issue.
But not = FreeBSD 12, I run the head.


Thank you everyone who = responded. It seems that the problem was with some BIOS configuration = changes, probably related to the power settings.
Once I reset everything to = factory defaults (plus some minimum "safe" and well-understood changes) = the problem went away.
It's really surprising that I saw it only with bhyve and only = with the particular kind of VMs. Perhaps there was a workload pattern = that triggered a hardware bug or overloaded some specific = module.

Anyways, = sorry for the noise and thank you for the help.

-- 
Andriy Gapon


https://standforukraine.com
https://razomforukraine.org

= --Apple-Mail=_FA2EEDB9-6782-4E0D-9AFD-58F553637AE0--