From nobody Sun Jan 02 23:35:39 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 63DC1193EBE6; Sun, 2 Jan 2022 23:35:44 +0000 (UTC) (envelope-from se@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JRwH41TVSz4SRj; Sun, 2 Jan 2022 23:35:44 +0000 (UTC) (envelope-from se@freebsd.org) Received: from [IPV6:2003:cd:5f26:900:608a:29ae:bdf5:24e5] (p200300cd5f260900608a29aebdf524e5.dip0.t-ipconnect.de [IPv6:2003:cd:5f26:900:608a:29ae:bdf5:24e5]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: se/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 1C1BA7DE5; Sun, 2 Jan 2022 23:35:43 +0000 (UTC) (envelope-from se@freebsd.org) Message-ID: <64aa958f-7a17-b504-b414-bdcbe7da5f51@freebsd.org> Date: Mon, 3 Jan 2022 00:35:39 +0100 List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.4.1 Subject: Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations Content-Language: en-US To: Konstantin Belousov Cc: Antoine Brodin , Konstantin Belousov , src-committers , dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org, FreeBSD Ports Management Team References: <202112301154.1BUBsR1q017491@gitrepo.freebsd.org> <9dffb50a-9374-be91-8007-ce8933571398@freebsd.org> From: Stefan Esser In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------dWcSTMnRWBk566yLqYyuhRR2" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1641166544; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=SV8ygxp0Dqxv3RYUUQ23fUFYh6jNua0QNITBYSGSBd0=; b=Y1wYKCLHMQMusXGvuLg5mYuVK3Ok6/XaCTheY0wuA4j5Bz1h0zS6T2mOO6r0ECAb1DUvAE IzCrwiXHms+bgQxCv2d5/Hhf1z2NU8YBWBQ91tPGjEjmvA6/0nRa2amJ5fdgGeoliqG1Ue 9xGp96JeROZRyNmz2isF//QKk6eeEBaM64LAIeV6ueHSfO6zIVH2jJgx5tM/ZUGdWoAIIs UhsKgsxHxVQxf7vtN+dpKsE9Ty4XcNB18QVaSaCnuzo1U6lRvjxT/XY4ADIb+6tZN3o/8y TkhBW0xA5SId1EcNITXZLgJfFlk2tLyIIRCyKmZlcR2RJlHPsp+ZSWFC6Nhncw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1641166544; a=rsa-sha256; cv=none; b=UsLkSbdaaUG/37F5Awkb2gWmZ0ibt+A/BmuViqSANwAZcQQIppg0vDGqRTg3Qh1w8SaWiW 5HN1vyZ3rxEfUOlTZXncdLw8STrCVjftHaC3QLwToqJKHely1ON6EZVWtbjlKzRx73vzPW Ayu5yDYH0pljgMYtNhvjL+6JviCHND7RNYHBcqPBK56wuQAxP5ymKJFP3HzeVcGDBXbTKy wHoOvuU0HJe61TU0gWjE5unQT8akosllAbVvyUaOrqT2N0Op6LC3W7FmfxIYjv1oz/dlJb cCMImCScIpqdLDYaWU1zojJfXC2ekxJFTKfaxXH0iS5Yw1G/au3ysRI46pbH+g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------dWcSTMnRWBk566yLqYyuhRR2 Content-Type: multipart/mixed; boundary="------------j1mDdjO0u8khgCaAYp0y9vbn"; protected-headers="v1" From: Stefan Esser To: Konstantin Belousov Cc: Antoine Brodin , Konstantin Belousov , src-committers , dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org, FreeBSD Ports Management Team Message-ID: <64aa958f-7a17-b504-b414-bdcbe7da5f51@freebsd.org> Subject: Re: git: e2650af157bc - main - Make CPU_SET macros compliant with other implementations References: <202112301154.1BUBsR1q017491@gitrepo.freebsd.org> <9dffb50a-9374-be91-8007-ce8933571398@freebsd.org> In-Reply-To: --------------j1mDdjO0u8khgCaAYp0y9vbn Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Am 02.01.22 um 23:16 schrieb Konstantin Belousov: > On Sun, Jan 02, 2022 at 10:45:14PM +0100, Stefan Esser wrote: >> Am 02.01.22 um 20:51 schrieb Antoine Brodin: [...] >> Python 3.8.12 (default, Dec 31 2021, 10:50:47) >>>>> import os >>>>> os.sched_getaffinity(0) >> Traceback (most recent call last): >> File "", line 1, in >> OSError: [Errno 34] Result too large >> >> This is a Python interpreter problem: it seems that the wrapper >> for the sched_getaffinity() function that has been introduced by >> kib in is buggy. >> >> As a work-around I have added a patch to comment out the >> os.sched_getaffinity(0) call (which used to cause an Attribute >> error that was caught by try/except, before). >> >> See ports commit 507c189b2876. >=20 > Buggy in which way? My assumption was that the wrapper in the Python interpreter in Modules/posixmodules.c function os_sched_getaffinity_impl() does not work with the FreeBSD implementation of sched_getaffinity(). The relevant code in the Python wrapper is: ncpus =3D NCPUS_START; while (1) { setsize =3D CPU_ALLOC_SIZE(ncpus); mask =3D CPU_ALLOC(ncpus); if (mask =3D=3D NULL) return PyErr_NoMemory(); if (sched_getaffinity(pid, setsize, mask) =3D=3D 0) break; CPU_FREE(mask); if (errno !=3D EINVAL) return posix_error(); if (ncpus > INT_MAX / 2) { PyErr_SetString(PyExc_OverflowError, "could not allocate " "a large enough CPU set"); return NULL; } ncpus =3D ncpus * 2; } NCPUS_START is 8 * sizeof(unsigned long) =3D 64 on a 64 bit CPU. > Our cpuset_getaffinity(2) syscall returns ERANGE for cpuset size not > equal to CPU_SETSIZE. It seems that python source expects EINVAL in > this case. Yes, anything except EINVAL will cause the loop to exit prematurely. > I can change the wrapper to translate ERANGE to EINVAL. sched_setaffin= ity() > probably would require a symmetrical patch, but lets postpone it. Yes. > diff --git a/lib/libc/gen/sched_getaffinity.c b/lib/libc/gen/sched_geta= ffinity.c > index 2ae8c5b763a3..8748d7a60278 100644 > --- a/lib/libc/gen/sched_getaffinity.c > +++ b/lib/libc/gen/sched_getaffinity.c > @@ -26,11 +26,29 @@ > * SUCH DAMAGE. > */ > =20 > +#include > #include > +#include > =20 > int > sched_getaffinity(pid_t pid, size_t cpusetsz, cpuset_t *cpuset) > { > + /* > + * Be more Linux-compatible: > + * - return EINVAL in passed size is less than size of cpuset_t > + * in advance, instead of ERANGE from the syscall > + * - if passed size is larger than the size of cpuset_t, be > + * permissive by claming it back to sizeof(cpuset_t) and > + * zeroing the rest. > + */ > + if (cpusetsz < sizeof(cpuset_t)) > + return (EINVAL); > + if (cpusetsz > sizeof(cpuset_t)) { > + memset((char *)cpuset + sizeof(cpuset_t), 0, > + cpusetsz - sizeof(cpuset_t)); > + cpusetsz =3D sizeof(cpuset_t); > + } > + > return (cpuset_getaffinity(CPU_LEVEL_WHICH, CPU_WHICH_PID, > pid =3D=3D 0 ? -1 : pid, cpusetsz, cpuset)); > } I have rebuilt the C library with this patch, but it did not fix the problem, since the value checked in the loop is errno, not the return code of sched_getaffinity(). The following code is tested to work: #include #include int sched_getaffinity(pid_t pid, size_t cpusetsz, cpuset_t *cpuset) { int result; result =3D cpuset_getaffinity(CPU_LEVEL_WHICH, CPU_WHICH_PID, pid =3D=3D 0 ? -1 : pid, cpusetsz, cpuset); if (result && errno =3D=3D ERANGE) errno =3D EINVAL; return (result); } Regards, STefan --------------j1mDdjO0u8khgCaAYp0y9vbn-- --------------dWcSTMnRWBk566yLqYyuhRR2 Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEEo3HqZZwL7MgrcVMTR+u171r99UQFAmHSNssFAwAAAAAACgkQR+u171r99US7 Xgf/SCx57At7J5uJsebLvOQrNHTJ232ARcmr5g/ZiFx/ORKUBf0xRv3PcvWs+era3vfmxPLj7LL6 GNLB6Bw20UJrOFsU4/4UoLIymeRqYfr5x4L3XBnlcNutey2AEnkU3fgVtEcx6KjuIdxHJXPNqrnI iFFk6siG5Uy/6wPv2pxNO8xu7SFhWgSM7I7m0//Mua9+qkd1tmKUfyhx+6tYNpkkhWXJAxTAn4e6 K0S9sK16BXIZco7ZwtuqzbVZV6uV7JEfdlKG05mDOpc1HpOrb8cNYjb83M+EzmJ+9c2NH2eLr4sO FEe7tu23g7yvf/BSCnw3okTJ1ywebB0aoyQqFMB9eg== =EJNb -----END PGP SIGNATURE----- --------------dWcSTMnRWBk566yLqYyuhRR2--