From nobody Mon May 30 09:22:40 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 11D101B55DFF; Mon, 30 May 2022 09:22:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LBVLS72qZz4dTm; Mon, 30 May 2022 09:22:40 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1653902561; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=LKJUzwgu73btNFbYo0FCmnBwEWclnARSeTV79sSBlX0=; b=GQlycyqZ0aXhAnDG/0idVrL8u6p+OCWxGNR8fkUXMP2YrlcCpDcw5+MYXVBJv1npSbWUgr FcttWopgBeOzXslWqsrOfZXwSeD/bKEGwXI2kEyVvw36Gt2JTN6+bfdxlYokIFsf8FuDSQ lr9+K+Z9msTsFGNtIGx1JJUOC/k2K8sl2EL8aVNc/h6S2cSOFz6ssrGyTKf70N8bCiArp6 h5Sg88+yR33eFf9PNHlSg3JDVdC6uzSp7zJD/1cowE5ijcx7SpaIWl00Aajt02cjdkxJ+g aolRDCbkFNRctGHEYZVlICF2lryI6D2s01g+FFKnfXGC9p4zVY9JOLocdqRzUw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C8B3E14B3E; Mon, 30 May 2022 09:22:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 24U9MeJB045902; Mon, 30 May 2022 09:22:40 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 24U9Meeh045901; Mon, 30 May 2022 09:22:40 GMT (envelope-from git) Date: Mon, 30 May 2022 09:22:40 GMT Message-Id: <202205300922.24U9Meeh045901@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Emmanuel Vadot Subject: git: 7468332f5518 - main - x86/mp: don't create empty cpu groups List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: manu X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 7468332f5518c1a725cd0067c35490f82ef781bd Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1653902561; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=LKJUzwgu73btNFbYo0FCmnBwEWclnARSeTV79sSBlX0=; b=IEZHnYREX7OdgTyXenlnEe+y9KEAABmQ50yrMeAK5MneHXsK72cOntDFZPfi7HGZ8YjwZ5 s9s5kZfLCwKJZaPn9I6PRl6zg6RuhZ9Sxw5GCTRopWKW8rcAkMMazMDASC/z3DyyzdhPQA agDv2EJYKJp4aTlxZDPKT04ocWAEoc5Wln/0wz8VZ23uCeqyB2xEI4aSdWNDhA7kMsoTz1 hrDE8JSwuKJEYP3V9iY8cmrI09gNS+egfnOxt53mvOnINGPnkrL1WDNnRc1C1zqoJ5h9Uy MRLUe88GvmonRnTgIphMKbnynIrFzKrQwzfnRZhoGge37Ge8GVwB7OxGd72fdA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1653902561; a=rsa-sha256; cv=none; b=jfb6eA3kW2XfsCswzRyEO9bDz5X8QltHka88xTUCUxS3PDZdp1IYXxdsuTk2muKvLQeSes eafyAWACoKgEDJPfdtQSXFbdb1li2Cumx6p3GUHUAGKwXKr7CeOsd0YQ/mp4hzkptDra13 JMqbIOSPhFk8qJjREMXnE2iuhIbYjqkz/UXlxCY4cSGnDo6zwRKeaVgyfFTeZ1w+dA/KhB uq9iPrUbDZckjGKWilqHimVesn/PdYEyN6qTLMToWE3cvaPcTN59X+RXQLgBsemtXJYtvl o36/4OxROsJh19/FnAActRJRETUoA/SkKFkSw5RB3JNPy1wJQ02/7XU2Xnpi8A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by manu: URL: https://cgit.FreeBSD.org/src/commit/?id=7468332f5518c1a725cd0067c35490f82ef781bd commit 7468332f5518c1a725cd0067c35490f82ef781bd Author: Corvin Köhne AuthorDate: 2022-05-30 09:19:14 +0000 Commit: Emmanuel Vadot CommitDate: 2022-05-30 09:21:46 +0000 x86/mp: don't create empty cpu groups When some APICs are disabled by tunables, some cpu groups could end up empty. An empty cpu group causes the system to panic because not all functions handle them correctly. Additionally, it's wasted time to handle and inspect empty cpu groups. Therefore, just don't create them. Reviewed by: kib, avg, cem Sponsored by: Beckhoff Automation GmbH & Co. KG MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D24927 --- sys/x86/x86/mp_x86.c | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/sys/x86/x86/mp_x86.c b/sys/x86/x86/mp_x86.c index 8ac5e4b4ddc2..3ca11700f2f2 100644 --- a/sys/x86/x86/mp_x86.c +++ b/sys/x86/x86/mp_x86.c @@ -871,6 +871,25 @@ x86topo_add_sched_group(struct topo_node *root, struct cpu_group *cg_root) nchildren = 0; node = root; while (node != NULL) { + /* + * When some APICs are disabled by tunables, nodes can end up + * with an empty cpuset. Nodes with an empty cpuset will be + * translated into cpu groups with empty cpusets. smp_topo_fill + * will then set cg_first and cg_last to -1. This isn't + * correctly handled in all functions. E.g. when + * cpu_search_lowest and cpu_search_highest loop through all + * cpus, they call CPU_ISSET on cpu -1 which ends up in a + * general protection fault. + * + * We could fix the scheduler to handle empty cpu groups + * correctly. Nevertheless, empty cpu groups are causing + * overhead for no value. So, it makes more sense to just don't + * create them. + */ + if (CPU_EMPTY(&node->cpuset)) { + node = topo_next_node(root, node); + continue; + } if (CPU_CMP(&node->cpuset, &root->cpuset) == 0) { if (node->type == TOPO_TYPE_CACHE && cg_root->cg_level < node->subtype) @@ -896,8 +915,14 @@ x86topo_add_sched_group(struct topo_node *root, struct cpu_group *cg_root) if (nchildren == root->cpu_count) return; - cg_root->cg_child = smp_topo_alloc(nchildren); + /* + * We are not interested in nodes without children. + */ cg_root->cg_children = nchildren; + if (nchildren == 0) + return; + + cg_root->cg_child = smp_topo_alloc(nchildren); /* * Now find again the same cache nodes as above and recursively @@ -909,7 +934,8 @@ x86topo_add_sched_group(struct topo_node *root, struct cpu_group *cg_root) if ((node->type != TOPO_TYPE_GROUP && node->type != TOPO_TYPE_NODE && node->type != TOPO_TYPE_CACHE) || - CPU_CMP(&node->cpuset, &root->cpuset) == 0) { + CPU_CMP(&node->cpuset, &root->cpuset) == 0 || + CPU_EMPTY(&node->cpuset)) { node = topo_next_node(root, node); continue; }