From nobody Wed Dec 21 18:34:02 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Ncht22tw5z1Ghpg; Wed, 21 Dec 2022 18:34:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Ncht22MxZz40gC; Wed, 21 Dec 2022 18:34:02 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1671647642; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ZGnZzJhkC9f1SjzYepShHy2lctXzg/enSk7hEs2Vc6I=; b=pAzqEi25gRoPlXXCi6HoOoPJA72bpNBEA6s9Wv2UsjY2Zvh/0Ike39CGab3kdGqpnrHUrW NvjDDI9BGLkqMkzHnp16+ZCCfmNz9rhv/f8aASl7fhXAl4W8KFg+xFGeonr3YWWoF/GfPd KnErxVIyTLvqh21BPF77oL+IEfadmS+0rnoiGI5Kzm3z88kaxJSdZqGJg0g+wso0gcIAiL VCDt5aAwl2H4kH99NmW0vSNpPyTTzMdA+cXWclIHHKUgt22mGZ6Q0fWmBSLiH7k1YZLciu 7MILCBLaXcRLxrDRYG4/orw7bjAQ3g8Zgi9bdTdcJ7v0oNywS4Dugnx0QnCmAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1671647642; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ZGnZzJhkC9f1SjzYepShHy2lctXzg/enSk7hEs2Vc6I=; b=Rm+iaePA4Ut8KRaAYRufGLj+mvp2wjkRwXiiTpz47nlRogfoVezWYcBRbsyGY9+0mM0Mdb 4Fv/Xv3HRh0O7eubvZQ8eYIYcuNXShf7+HgWexV5CyjrVEqHbQe3wS0EpdAYhVtYyPzgNt p0aKGzyoV6Jv6cmP6O1VjRwzMs6LwTlmHnMu9+7Ol6JY326fmeJfvTV300KpwyeXcG//i0 Dl5REpQyceLFThpKo6aJIEIJd8xUFRSEc55x85iN0Asy0Ob5cXXo71OoaFhbuoJHmwyWRV I62fkBzysg8e75N8oHHt58WGgbyAQhTatwUxd7IyQt5loNZNcOHslFWVE3RBHQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1671647642; a=rsa-sha256; cv=none; b=APCm2czB8CHK34uXIQcUT2Lrs2z9TXRWBaDJujgNrNI5l4zn+ZLOwAUgVXqE3UAqx0LKYz y18n7x0o9di599u/Q03CNZqS9IVxB96xqi8kB5aCH7fQXLd7WiZzVuF8q3QfcINt2K7SaS c4bm5odegZEQHDjVv40kbe0ukSESu9pGbjio5ufEo9LMN5vVBcHE6+nuoWjIi2ubzZAZS3 r0AvgbiSzY5twuYdhnzImKOj4xs0jvitQCCggAXyS9fyWV8yTTkLHb7i34OR2uFwHbnHoh +oi/F95whdwHEW/dEVtWrPIZ5Yiv75A5Lg8ggnRK7pfizWgxuJGHZJ8CJucjkw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Ncht21TYVzQLt; Wed, 21 Dec 2022 18:34:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 2BLIY2v6068844; Wed, 21 Dec 2022 18:34:02 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 2BLIY2Lb068843; Wed, 21 Dec 2022 18:34:02 GMT (envelope-from git) Date: Wed, 21 Dec 2022 18:34:02 GMT Message-Id: <202212211834.2BLIY2Lb068843@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: 461663ddbad0 - main - bhyve: Simplify setting vCPU capabilities. List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 461663ddbad02a4a5135673d545695b1a9f25ed0 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=461663ddbad02a4a5135673d545695b1a9f25ed0 commit 461663ddbad02a4a5135673d545695b1a9f25ed0 Author: John Baldwin AuthorDate: 2022-12-21 18:31:16 +0000 Commit: John Baldwin CommitDate: 2022-12-21 18:31:16 +0000 bhyve: Simplify setting vCPU capabilities. - Enable VM_CAP_IPI_EXIT in fbsdrun_set_capabilities along with other capabilities enabled on all vCPUs. - Don't call fbsdrun_set_capabilities a second time on the BSP in spinup_vcpu. - To preserve previous behavior, don't unconditionally enable unrestricted guest mode on the BSP (this unbreaks single-vCPU guests on Nehalem systems, though supporting such setups is of dubious value). Other places that enbale UG on the BSP are careful to check the result of the operation and fail if it is not available. - Don't set any capabilities in spinup_ap(). These are now all redundant with earlier settings from spinup_vcpu(). - While here, axe a stale comment from fbsdrun_addcpu(). This function is now always called from the main thread for all vCPUs. Reviewed by: corvink, markj Differential Revision: https://reviews.freebsd.org/D37642 --- usr.sbin/bhyve/bhyverun.c | 28 +++++++++++++++------------- usr.sbin/bhyve/bhyverun.h | 1 - usr.sbin/bhyve/spinup_ap.c | 14 -------------- 3 files changed, 15 insertions(+), 28 deletions(-) diff --git a/usr.sbin/bhyve/bhyverun.c b/usr.sbin/bhyve/bhyverun.c index 8bc419504771..72f806e97993 100644 --- a/usr.sbin/bhyve/bhyverun.c +++ b/usr.sbin/bhyve/bhyverun.c @@ -552,12 +552,6 @@ fbsdrun_addcpu(struct vmctx *ctx, int newcpu, uint64_t rip, bool suspend) { int error; - /* - * The 'newcpu' must be activated in the context of 'fromcpu'. If - * vm_activate_cpu() is delayed until newcpu's pthread starts running - * then vmm.ko is out-of-sync with bhyve and this can create a race - * with vm_suspend(). - */ error = vm_activate_cpu(ctx, newcpu); if (error != 0) err(EX_OSERR, "could not activate CPU %d", newcpu); @@ -1044,7 +1038,7 @@ num_vcpus_allowed(struct vmctx *ctx) return (1); } -void +static void fbsdrun_set_capabilities(struct vmctx *ctx, int cpu) { int err, tmp; @@ -1086,6 +1080,9 @@ fbsdrun_set_capabilities(struct vmctx *ctx, int cpu) } vm_set_capability(ctx, cpu, VM_CAP_ENABLE_INVPCID, 1); + + err = vm_set_capability(ctx, cpu, VM_CAP_IPI_EXIT, 1); + assert(err == 0); } static struct vmctx * @@ -1157,14 +1154,19 @@ spinup_vcpu(struct vmctx *ctx, int vcpu, bool suspend) int error; uint64_t rip; - error = vm_get_register(ctx, vcpu, VM_REG_GUEST_RIP, &rip); - assert(error == 0); + if (vcpu != BSP) { + fbsdrun_set_capabilities(ctx, vcpu); - fbsdrun_set_capabilities(ctx, vcpu); - error = vm_set_capability(ctx, vcpu, VM_CAP_UNRESTRICTED_GUEST, 1); - assert(error == 0); + /* + * Enable the 'unrestricted guest' mode for APs. + * + * APs startup in power-on 16-bit mode. + */ + error = vm_set_capability(ctx, vcpu, VM_CAP_UNRESTRICTED_GUEST, 1); + assert(error == 0); + } - error = vm_set_capability(ctx, vcpu, VM_CAP_IPI_EXIT, 1); + error = vm_get_register(ctx, vcpu, VM_REG_GUEST_RIP, &rip); assert(error == 0); fbsdrun_addcpu(ctx, vcpu, rip, suspend); diff --git a/usr.sbin/bhyve/bhyverun.h b/usr.sbin/bhyve/bhyverun.h index 1e1e5f9880f8..d3eb8c8b23da 100644 --- a/usr.sbin/bhyve/bhyverun.h +++ b/usr.sbin/bhyve/bhyverun.h @@ -45,7 +45,6 @@ void *paddr_guest2host(struct vmctx *ctx, uintptr_t addr, size_t len); uintptr_t paddr_host2guest(struct vmctx *ctx, void *addr); #endif -void fbsdrun_set_capabilities(struct vmctx *ctx, int cpu); int fbsdrun_virtio_msix(void); int vmexit_task_switch(struct vmctx *, struct vm_exit *, int *vcpu); diff --git a/usr.sbin/bhyve/spinup_ap.c b/usr.sbin/bhyve/spinup_ap.c index e23265f8b82b..af8d40030a08 100644 --- a/usr.sbin/bhyve/spinup_ap.c +++ b/usr.sbin/bhyve/spinup_ap.c @@ -87,20 +87,6 @@ spinup_ap(struct vmctx *ctx, int newcpu, uint64_t rip) error = vcpu_reset(ctx, newcpu); assert(error == 0); - fbsdrun_set_capabilities(ctx, newcpu); - - /* - * Enable the 'unrestricted guest' mode for 'newcpu'. - * - * Set up the processor state in power-on 16-bit mode, with the CS:IP - * init'd to the specified low-mem 4K page. - */ - error = vm_set_capability(ctx, newcpu, VM_CAP_UNRESTRICTED_GUEST, 1); - assert(error == 0); - - error = vm_set_capability(ctx, newcpu, VM_CAP_IPI_EXIT, 1); - assert(error == 0); - spinup_ap_realmode(ctx, newcpu, &rip); vm_resume_cpu(ctx, newcpu);