From nobody Sat Nov 13 12:21:01 2021 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 173AE1847931; Sat, 13 Nov 2021 12:21:13 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hrvgr50qzz4Vm8; Sat, 13 Nov 2021 12:21:12 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 1ADCL10N072632 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 13 Nov 2021 14:21:04 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 1ADCL10N072632 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 1ADCL1Pg072631; Sat, 13 Nov 2021 14:21:01 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 13 Nov 2021 14:21:01 +0200 From: Konstantin Belousov To: Antoine Brodin Cc: src-committers , dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: 160b4b922b60 - main - Add real sched.h Message-ID: References: <202111101934.1AAJYJFQ041545@gitrepo.freebsd.org> List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on tom.home X-Rspamd-Queue-Id: 4Hrvgr50qzz4Vm8 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Sat, Nov 13, 2021 at 08:48:36AM +0000, Antoine Brodin wrote: > On Thu, Nov 11, 2021 at 8:06 AM Konstantin Belousov wrote: > > > > On Thu, Nov 11, 2021 at 07:29:19AM +0000, Antoine Brodin wrote: > > > On Wed, Nov 10, 2021 at 7:34 PM Konstantin Belousov wrote: > > > > > > > > The branch main has been updated by kib: > > > > > > > > URL: https://cgit.FreeBSD.org/src/commit/?id=160b4b922b6021848b6b48afc894d16b879b7af2 > > > > > > > > commit 160b4b922b6021848b6b48afc894d16b879b7af2 > > > > Author: Konstantin Belousov > > > > AuthorDate: 2021-10-22 15:17:21 +0000 > > > > Commit: Konstantin Belousov > > > > CommitDate: 2021-11-10 19:18:53 +0000 > > > > > > > > Add real sched.h > > > > > > > > It is required by IEEE Std 1003.1-2008 AKA POSIX. > > > > > > > > Put some Linux compatibility stuff under BSD_VISIBLE namespace, in > > > > particular, sys/cpuset.h definitions. Also, if user really want > > > > Linux compatibility, she can request cpu_set_t typedef with > > > > _WITH_CPU_SET_T define. > > > > > > > > Reviewed by: jhb > > > > Sponsored by: The FreeBSD Foundation > > > > MFC after: 1 week > > > > Differential revision: https://reviews.freebsd.org/D32901 > > > > --- > > > > include/Makefile | 5 +++-- > > > > include/sched.h | 46 ++++++++++++++++++++++++++++++++++++++++++++++ > > > > 2 files changed, 49 insertions(+), 2 deletions(-) > > > > > > Hi, > > > > > > It seems that this broke some ports: > > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/ninja-1.10.2,2.log > > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/ghc-8.10.7.log > > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/openpgm-5.2.122_6.log > > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/guile2-2.2.7_1.log > > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/liburcu-0.12.1.log > > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/guile-3.0.7.log > > > http://gohan03.nyi.freebsd.org/data/main-amd64-default-baseline/p913ccecf2a92_s448bcd01dc/logs/errors/squid-4.15.log > > > ... > > > > Could you please try this? My intent was that cpu_set_t would be requested > > as needed by consumers with _WITH_CPU_SET_T. > > > > commit b6bb08f09dc7cf2796c35015f5aee93dedf33018 > > Author: Konstantin Belousov > > Date: Thu Nov 11 10:01:54 2021 +0200 > > > > sched.h: Make cpu_set_t compat typedef visible unconditionally > > > > or rather, visible if BSD namespace is enabled, instead of requiring > > _WITH_CPU_SET_T define to expose it. It seems a lot of code assumes > > that presence of sched_getaffinity() implies existence of cpu_set_t. > > > > Reported by: portmgr (antoine) > > Sponsored by: The FreeBSD Foundation > > MFC after: 1 week > > Hello, > > With this patch, this is better (less broken ports). > But this seems to expose problems with other headers ( sys/bitset.h > for instance) > > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/xplanet-1.3.1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mixxx22-2.2.4_7.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/Sorcer-lv2-1.1.3_8.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/polygraph-4.13.0_2.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/stress-ng-0.13.06.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/abyss-2.3.1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/gcta-1.26.0_3.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/gemma-0.98.3_1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/iqtree-2.0.6_2.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/ncbi-blast+-2.12.0_1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/ncbi-cxx-toolkit-25.2.0.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/rna-STAR-2.7.3.a_1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mysql57-server-5.7.36.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mysql80-server-8.0.26_1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mysqlwsrep57-server-5.7.34.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/percona57-pam-for-mysql-5.7.34.37.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/percona57-server-5.7.34.37.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/maliit-framework-2.1.1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/electron12-12.2.2.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/kdevelop-pg-qt-2.2.1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/libvirt-7.9.0.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/util-linux-2.36_2.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/tea-60.5.1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/jzintv-1.0.20200712,1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/libc6-shim-20210627.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mame-0.226_1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mess-0.226_1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/keeperrl-0.0.23_5.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/RetroArch-1.9.7.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/stonesoup-sdl-0.27.1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/stonesoup-0.27.1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mypaint-2.0.0_3.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/gcc11-11.2.0.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/gcc11-devel-11.2.1.s20211009.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/gcc12-devel-12.0.0.s20211010.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mono-5.10.1.57_3.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mono5.10-5.10.1.57_3.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mono5.20-5.20.1.34.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/mono6.8-6.8.0.123.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-4.94.2_1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-ldap2-4.94.2_1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-monitor-4.94.2_1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-mysql-4.94.2_1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-postgresql-4.94.2_1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-sa-exim-4.94.2+4.2.1_1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/exim-sqlite-4.94.2_1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/giacxcas-1.6.0.7_3.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/libnormaliz-3.9.0_1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/libmediainfo-21.09.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/pipewire-0.3.36_1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/aircrack-ng-1.5.2_2.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/ettercap-0.8.3.1,1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/glusterfs-8.4.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/nepenthes-0.2.2_16.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/relayd-5.5.20140810_8.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/yate-6.4.0_1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/miktex-21.8.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/bagel-1.2.2_6.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/qbox-1.73.3.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/openssh-portable-hpn-8.8.p1_1,1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/openssh-portable-8.8.p1_1,1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/flock-2.30.1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/intel-pcm-202109.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/intel-pcm-devel-g20211013,1.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/zbackup-1.4.4_19.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/apt-cacher-ng-3.2_2.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/node10-10.24.1_2.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/proxygen-2021.11.01.00.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/squid-4.15.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/squid-devel-5.0.6.log > http://gohan04.nyi.freebsd.org/data/main-amd64-sched-default/2021-11-11_09h04m08s/logs/errors/vdr-plugin-live-0.3.0_5.log Sigh. So ut cannot work this way. Of course I can (and probably should) implement missing Linux CPU_* macros, but BIT_* macros have incompatible API. There is no other way than requiring each program to opt-in by defining _WITH_CPU_SET_T, instead of trying to be fully Linux-compatible there. I will commit the following patch after buildworld finishes. I am not sure if it makes sense to do dedicated exp-run to see if I missed something, or just wait for regular run to finish with updated world. In any case, please ping me. commit 49c92416389668ab1b4c7cb37d4bdf4545369a58 Author: Konstantin Belousov Date: Thu Nov 11 10:01:54 2021 +0200 sched.h: Hide all Linux compat sched_* functions under _WITH_CPU_SET_T Instead of only hiding cpu_set_t compat typedef. Too many software packages assumes that sched_getaffinity() presence implies full source compatibility with glibc. We can (and should) handle missing CPU_* macros, but there are incompatible BIT_* uses which cannot be fixed in src/. So hide everything under _WITH_CPU_SET_T, in particular, do not expose sched_get/setaffinity(), as well as CPU_* and BIT_* macros. Consumers that want sched* functions must opt-in. Reported by: portmgr (antoine) Sponsored by: The FreeBSD Foundation MFC after: 1 week diff --git a/include/sched.h b/include/sched.h index 1c41cda79046..460d3bfed154 100644 --- a/include/sched.h +++ b/include/sched.h @@ -32,12 +32,9 @@ #include #include #include -#if __BSD_VISIBLE -#include -#endif /* __BSD_VISIBLE */ - #if __BSD_VISIBLE #ifdef _WITH_CPU_SET_T +#include struct _cpuset; typedef struct _cpuset cpu_set_t; #endif /* _WITH_CPU_SET_T */ @@ -45,9 +42,11 @@ typedef struct _cpuset cpu_set_t; __BEGIN_DECLS #if __BSD_VISIBLE +#ifdef _WITH_CPU_SET_T int sched_getaffinity(pid_t pid, size_t cpusetsz, cpuset_t *cpuset); int sched_setaffinity(int pid, size_t cpusetsz, const cpuset_t *cpuset); int sched_getcpu(void); +#endif /* _WITH_CPU_SET_T */ #endif /* __BSD_VISIBLE */ __END_DECLS diff --git a/lib/libc/gen/sched_getaffinity.c b/lib/libc/gen/sched_getaffinity.c index 191374a3c5e3..5557d3d93b47 100644 --- a/lib/libc/gen/sched_getaffinity.c +++ b/lib/libc/gen/sched_getaffinity.c @@ -26,6 +26,7 @@ * SUCH DAMAGE. */ +#define _WITH_CPU_SET_T #include int diff --git a/lib/libc/gen/sched_getcpu_gen.c b/lib/libc/gen/sched_getcpu_gen.c index ef1bebc2915a..704075c0ae53 100644 --- a/lib/libc/gen/sched_getcpu_gen.c +++ b/lib/libc/gen/sched_getcpu_gen.c @@ -26,6 +26,7 @@ * SUCH DAMAGE. */ +#define _WITH_CPU_SET_T #include #include "libc_private.h" diff --git a/lib/libc/gen/sched_setaffinity.c b/lib/libc/gen/sched_setaffinity.c index 9368b1752b74..ad775b5dbce5 100644 --- a/lib/libc/gen/sched_setaffinity.c +++ b/lib/libc/gen/sched_setaffinity.c @@ -26,6 +26,7 @@ * SUCH DAMAGE. */ +#define _WITH_CPU_SET_T #include int diff --git a/lib/libc/x86/sys/sched_getcpu_x86.c b/lib/libc/x86/sys/sched_getcpu_x86.c index 13ba18ef5e36..5e3bd5311734 100644 --- a/lib/libc/x86/sys/sched_getcpu_x86.c +++ b/lib/libc/x86/sys/sched_getcpu_x86.c @@ -32,6 +32,7 @@ #include #include #include +#define _WITH_CPU_SET_T #include #include "libc_private.h"