From nobody Thu Sep 29 20:20:01 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Mdl8d56rmz4V7w3; Thu, 29 Sep 2022 20:20:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Mdl8d4cSpz3FWs; Thu, 29 Sep 2022 20:20:01 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1664482801; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=yDT7kUGhrPvHWPc143+3cCnb7dwRxJTWI7AA3HrWGCc=; b=INBVRuqIl3CJYR855q43uQOdMH5IizcBAkGFoxELWDxtld5XpAL8mM0/kQn2eEyYP2PEhV 4IXFPRKPAW3trdTP+qUdzjf3nR0bMDVnVda12tEDIKY3Sx/UOy3afEJ5nmfR4uBtlYctN4 CJgOqCa5m+sKoHdJ+nZOBnDHrQyTV12XwtEuPrlcR9qj6amhHNl/PSQytgt4Bp932Kxxox ab9ZzbVNpJfsvCA2emB71p93AVt4cYAoPZIoWW8Rrzb1nwtLs3Dxw5Okz656EOtt/26XZ6 P1KeN9y9/bz8isBktVwqRb1VPyAH1mibix8zugpOUv3g+olr3EYLX3nYpecp4A== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Mdl8d3g2CzwW2; Thu, 29 Sep 2022 20:20:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 28TKK1qM067639; Thu, 29 Sep 2022 20:20:01 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 28TKK119067626; Thu, 29 Sep 2022 20:20:01 GMT (envelope-from git) Date: Thu, 29 Sep 2022 20:20:01 GMT Message-Id: <202209292020.28TKK119067626@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Ed Maste Subject: git: d31a21b939f7 - stable/13 - x86/mp: don't create empty cpu groups List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: emaste X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: d31a21b939f7b1380f3055374317b71a034976a4 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1664482801; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=yDT7kUGhrPvHWPc143+3cCnb7dwRxJTWI7AA3HrWGCc=; b=DKP0v93KWSU3JL7sT1eBr4dAvZMRpzL6IUEr2/advnENLNum3IW7bpSB8g1Cig+IPLOfZp 1fwQp8vwTesS1JrIsL4iYd4Ogxli0qoY1oFlz5NhLxpX2rJ3eksg9HCc9VgVryjX8mCxKu +K8ADBpYCZQ50EkRJHs4EG/JG9RgY4MrxMmIT3Iwx4qTQSh0kN9YaC57DIyL7fE/p7WR0J djH+7/SFDqUqjXz4+OFtpLFGiyvOz4RDRMMLRQ4ikRM/bIF82LIfOepIhYb7z68spnjpJ4 GiDkoXj2AsPi1XmOlu1lYUiWSIUxKhzIJg0Idfn9WJNA8rCTqZvLhIZGlSqJDw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1664482801; a=rsa-sha256; cv=none; b=RkJ8rj7dGWDT4B1xOPp7b4opmWmhZNSEqeOlPCFe3gg5WQ2RxH8z27GEaoLaErGpt/1dTQ BgtFjbujZGrGeBNgMgaws7nRpKHTm+0SwDgv4fxkjjL+6HgtJ6ykluqV7fwfA9mN4nQmA7 l9nY4ObV0IsAjUzaDa44q3TQ2tWgY1mvseF7KjiNMLTqoDgbeY/ja3P+kpLrevF/geg1u9 o8gC5Tw5awgpHYDCQu++GQ5zjkY38mjYVafWzIgBf4rRuOQBsFthIVO1EGAScwzi0g2cql Fn0rnSWrjGOcB8Hc0W3R5HMMoPGLqVXbM30ZA2tRwHJvlgy1bNVU4VBEMplkQw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by emaste: URL: https://cgit.FreeBSD.org/src/commit/?id=d31a21b939f7b1380f3055374317b71a034976a4 commit d31a21b939f7b1380f3055374317b71a034976a4 Author: Corvin Köhne AuthorDate: 2022-05-30 09:19:14 +0000 Commit: Ed Maste CommitDate: 2022-09-29 19:58:50 +0000 x86/mp: don't create empty cpu groups When some APICs are disabled by tunables, some cpu groups could end up empty. An empty cpu group causes the system to panic because not all functions handle them correctly. Additionally, it's wasted time to handle and inspect empty cpu groups. Therefore, just don't create them. Reviewed by: kib, avg, cem Sponsored by: Beckhoff Automation GmbH & Co. KG MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D24927 (cherry picked from commit 7468332f5518c1a725cd0067c35490f82ef781bd) --- sys/x86/x86/mp_x86.c | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/sys/x86/x86/mp_x86.c b/sys/x86/x86/mp_x86.c index 289885fa6213..a7aa508f7699 100644 --- a/sys/x86/x86/mp_x86.c +++ b/sys/x86/x86/mp_x86.c @@ -871,6 +871,25 @@ x86topo_add_sched_group(struct topo_node *root, struct cpu_group *cg_root) nchildren = 0; node = root; while (node != NULL) { + /* + * When some APICs are disabled by tunables, nodes can end up + * with an empty cpuset. Nodes with an empty cpuset will be + * translated into cpu groups with empty cpusets. smp_topo_fill + * will then set cg_first and cg_last to -1. This isn't + * correctly handled in all functions. E.g. when + * cpu_search_lowest and cpu_search_highest loop through all + * cpus, they call CPU_ISSET on cpu -1 which ends up in a + * general protection fault. + * + * We could fix the scheduler to handle empty cpu groups + * correctly. Nevertheless, empty cpu groups are causing + * overhead for no value. So, it makes more sense to just don't + * create them. + */ + if (CPU_EMPTY(&node->cpuset)) { + node = topo_next_node(root, node); + continue; + } if (CPU_CMP(&node->cpuset, &root->cpuset) == 0) { if (node->type == TOPO_TYPE_CACHE && cg_root->cg_level < node->subtype) @@ -896,8 +915,14 @@ x86topo_add_sched_group(struct topo_node *root, struct cpu_group *cg_root) if (nchildren == root->cpu_count) return; - cg_root->cg_child = smp_topo_alloc(nchildren); + /* + * We are not interested in nodes without children. + */ cg_root->cg_children = nchildren; + if (nchildren == 0) + return; + + cg_root->cg_child = smp_topo_alloc(nchildren); /* * Now find again the same cache nodes as above and recursively @@ -909,7 +934,8 @@ x86topo_add_sched_group(struct topo_node *root, struct cpu_group *cg_root) if ((node->type != TOPO_TYPE_GROUP && node->type != TOPO_TYPE_NODE && node->type != TOPO_TYPE_CACHE) || - CPU_CMP(&node->cpuset, &root->cpuset) == 0) { + CPU_CMP(&node->cpuset, &root->cpuset) == 0 || + CPU_EMPTY(&node->cpuset)) { node = topo_next_node(root, node); continue; }