From nobody Fri Mar 17 12:50:30 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PdP9z0848z3yL8d; Fri, 17 Mar 2023 12:50:31 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PdP9y6jR9z3pfy; Fri, 17 Mar 2023 12:50:30 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1679057430; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=xUYvQJEkCKxgBsHrb6a02SBwKe4tVDB027jO5C6VRzo=; b=rxQeSBKQ7sAVuAh55Pe5SuzGjZDAnH2cocQkirr3h/x+t0ZY9btR8B998rjenojwN6SRYH ZcMFEbmokTlxdMM//v2Inf9Tato8lWHf71NWjPozx9ByOTW5ZTdX4oI53/m965SVGCvs2t KBOXcywDCDUEN5PpZoj93uHlcvyFRPL+9dma8InxxWPTrTM5/vwM9L7dJwxeybfrp8Qa5E 2EAnlrf8o5NkRG22cMtyqWf1VPaMiBctz8QUY2BBLeVr26fFowpwL55D1AqWpgMYFHbImT HTd5cdaWZhepBtxswJvVgqYtZ8hxq6jhBqUmbzyrmTlqP1EWj0EmBUdhJMIACQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1679057430; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=xUYvQJEkCKxgBsHrb6a02SBwKe4tVDB027jO5C6VRzo=; b=InSy7byDN1MG+lAQPsmMjX5gCfQwAPj3bEzmA9zhY5b0vqyYXh7+9cnkcUk3jVVtVTadeI +gqNFehhcWcrHzhnLwveTGic1y+Aqd7qkV0zIbqxHQmLRsgvNqO7TShtiCfUy0qAVeK7DJ ewlgEK3PvfqQ/ILT7Kxg7YZWgTu21bWoTmLPqDW0JdJZRZ7qvn2abeDGojfQYqWE82dBuS rcDDNx9hNJj4M0kYF4BnSdwcxfMOM/AVVSkNBctDtiH7hxOpMDCR+hwEZ4fXsD7SfwbQLu WfiVzuwGZByXR0jthFtg3S96aIIpy00/u585WiaWnePFJteKgeMB5PWKQlnSqA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1679057430; a=rsa-sha256; cv=none; b=U+lDlXlrtPbKzvwp3VqeUhvc7FEPkbvlbbJ2Mw7vJf/wxBLmahl/jSkNj9u5Nqkg9iCeFc 4+5PA6Jr4Rh6hvPwdTaVJfH96AH1YHtx+O025Mn+yn8Qg1LPG4AHnfLorjAalUmwv0UVzv mFDZm0aEjGoI4TLROFpsYieM2vKQ9944iUvRECLJXeHstMzCSD4tS3r6GO8MJ88VSOilVU vlr+Gl9nRHbUBC9A8cSCITJ18/zUwTdppcMD4gqQ2/f82g+xKP7B08QhBE2nqKX30q+zRv TtG232mXbdv7BMDMikae3F7jID3G0x8R7opg3glm2Q7KJE0MNTr2aqkHl9YJzA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4PdP9y5mL9zR11; Fri, 17 Mar 2023 12:50:30 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 32HCoUtE005580; Fri, 17 Mar 2023 12:50:30 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 32HCoULQ005579; Fri, 17 Mar 2023 12:50:30 GMT (envelope-from git) Date: Fri, 17 Mar 2023 12:50:30 GMT Message-Id: <202303171250.32HCoULQ005579@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: =?utf-8?Q?Corvin=20K=C3=B6hne?= Subject: git: 94a3876d7e18 - main - vmm: fix missing ipi statistic List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: corvink X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 94a3876d7e18ada9596464623829d37d186da856 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by corvink: URL: https://cgit.FreeBSD.org/src/commit/?id=94a3876d7e18ada9596464623829d37d186da856 commit 94a3876d7e18ada9596464623829d37d186da856 Author: Vitaliy Gusev AuthorDate: 2023-03-17 09:17:22 +0000 Commit: Corvin Köhne CommitDate: 2023-03-17 12:50:08 +0000 vmm: fix missing ipi statistic ipi counters are missing in bhyvectl's output because vm_maxcpu is 0 when initializing them. That's because vmm_stat_register is executed before vmm_init. Instead of directly fixing it, there's a better solution in illumos which is cherry picked: https://github.com/illumos/illumos-gate/commit/65a3bc83734e5fb0fc2c19df3e5112b87dcdc3f8 It replaces the matrix statistic by two counters per vcpu. One for counting the ipis to the vcpu and one counting the ipis received by the vcpu. This has several advantages: - A matrix statistic becomes huge when using many vcpus. - A matrix statistic easily reaches the MAX_VMM_STAT_ELEMS limit. - Two counters are enough in most cases. DTrace can be used for more advanced debugging purposes. - A matrix statistic wastes memory. The matrix size is determined by vm_maxcpu regardless of the number of vcpus assigned to the vm. Reviewed by: corvink, markj Fixes: ee98f99d7a68b284a669fefb969cbfc31df2d0ab ("vmm: Convert VM_MAXCPU into a loader tunable hw.vmm.maxcpu.") MFC after: 1 week Sponsored by: vStack Differential Revision: https://reviews.freebsd.org/D39038 --- sys/amd64/vmm/io/vlapic.c | 9 ++++++--- sys/amd64/vmm/vmm_stat.c | 3 --- sys/amd64/vmm/vmm_stat.h | 2 -- 3 files changed, 6 insertions(+), 8 deletions(-) diff --git a/sys/amd64/vmm/io/vlapic.c b/sys/amd64/vmm/io/vlapic.c index 884e232b1422..2144cbabd979 100644 --- a/sys/amd64/vmm/io/vlapic.c +++ b/sys/amd64/vmm/io/vlapic.c @@ -905,7 +905,8 @@ vlapic_calcdest(struct vm *vm, cpuset_t *dmask, uint32_t dest, bool phys, } } -static VMM_STAT_ARRAY(IPIS_SENT, VMM_STAT_NELEMS_VCPU, "ipis sent to vcpu"); +static VMM_STAT(VLAPIC_IPI_SEND, "ipis sent from vcpu"); +static VMM_STAT(VLAPIC_IPI_RECV, "ipis received by vcpu"); static void vlapic_set_tpr(struct vlapic *vlapic, uint8_t val) @@ -1102,7 +1103,8 @@ vlapic_icrlo_write_handler(struct vlapic *vlapic, bool *retu) CPU_FOREACH_ISSET(i, &dmask) { vcpu = vm_vcpu(vlapic->vm, i); lapic_intr_edge(vcpu, vec); - vmm_stat_array_incr(vlapic->vcpu, IPIS_SENT, i, 1); + vmm_stat_incr(vlapic->vcpu, VLAPIC_IPI_SEND, 1); + vmm_stat_incr(vcpu, VLAPIC_IPI_RECV, 1); VLAPIC_CTR2(vlapic, "vlapic sending ipi %d to vcpuid %d", vec, i); } @@ -1223,7 +1225,8 @@ vlapic_self_ipi_handler(struct vlapic *vlapic, uint64_t val) vec = val & 0xff; lapic_intr_edge(vlapic->vcpu, vec); - vmm_stat_array_incr(vlapic->vcpu, IPIS_SENT, vlapic->vcpuid, 1); + vmm_stat_incr(vlapic->vcpu, VLAPIC_IPI_SEND, 1); + vmm_stat_incr(vlapic->vcpu, VLAPIC_IPI_RECV, 1); VLAPIC_CTR1(vlapic, "vlapic self-ipi %d", vec); } diff --git a/sys/amd64/vmm/vmm_stat.c b/sys/amd64/vmm/vmm_stat.c index 2750982185aa..2df4423bc60f 100644 --- a/sys/amd64/vmm/vmm_stat.c +++ b/sys/amd64/vmm/vmm_stat.c @@ -70,9 +70,6 @@ vmm_stat_register(void *arg) if (vst->scope == VMM_STAT_SCOPE_AMD && !vmm_is_svm()) return; - if (vst->nelems == VMM_STAT_NELEMS_VCPU) - vst->nelems = vm_maxcpu; - if (vst_num_elems + vst->nelems >= MAX_VMM_STAT_ELEMS) { printf("Cannot accommodate vmm stat type \"%s\"!\n", vst->desc); return; diff --git a/sys/amd64/vmm/vmm_stat.h b/sys/amd64/vmm/vmm_stat.h index 050d3c13dda1..e40a960ec82a 100644 --- a/sys/amd64/vmm/vmm_stat.h +++ b/sys/amd64/vmm/vmm_stat.h @@ -58,8 +58,6 @@ struct vmm_stat_type { void vmm_stat_register(void *arg); -#define VMM_STAT_NELEMS_VCPU (-1) - #define VMM_STAT_FDEFINE(type, nelems, desc, func, scope) \ struct vmm_stat_type type[1] = { \ { -1, nelems, desc, func, scope } \