Re: git: c21b080f3dc2 - main - cpuset: Fix sched_[g|s]etaffinity() for better compatibility with Linux.

On Sun, Feb 12, 2023 at 11:13 AM Dmitry Chagin <dchagin@heemeyer.club> wrote:
>
> On Sun, Feb 12, 2023 at 01:40:59PM +0300, Dmitry Chagin wrote:
> > On Sun, Feb 12, 2023 at 09:47:15AM +0000, Antoine Brodin wrote:
> > > On Sun, Jan 29, 2023 at 1:18 PM Dmitry Chagin <dchagin@freebsd.org> wrote:
> > > >
> > > > The branch main has been updated by dchagin:
> > > >
> > > > URL: https://cgit.FreeBSD.org/src/commit/?id=c21b080f3dc2f5e91ada608d4385b7ed6538ba9b
> > > >
> > > > commit c21b080f3dc2f5e91ada608d4385b7ed6538ba9b
> > > > Author:     Dmitry Chagin <dchagin@FreeBSD.org>
> > > > AuthorDate: 2023-01-29 13:17:33 +0000
> > > > Commit:     Dmitry Chagin <dchagin@FreeBSD.org>
> > > > CommitDate: 2023-01-29 13:17:33 +0000
> > > >
> > > >     cpuset: Fix sched_[g|s]etaffinity() for better compatibility with Linux.
> > > >
> > > >     Under Linux to sched_[g|s]etaffinity() functions the value returned from a call
> > > >     to gettid(2) (thread id) can be passed in the argument pid. Specifying pid as 0
> > > >     will set the attribute for the calling thread, and passing the value returned
> > > >     from a call to getpid(2) (process id) will set the attribute for the main thread
> > > >     of the thread group.
> > > >
> > > >     Native cpuset(2) family of system calls has "which" argument to determine how
> > > >     the value of id argument is interpreted, i.e., CPU_WHICH_TID is used to pass
> > > >     a thread id and CPU_WHICH_PID - to pass a process id.
> > > >
> > > >     For now native sched_[g|s]etaffinity() implementation is wrong as uses "which"
> > > >     CPU_WHICH_PID to pass both (process and thread id) to the kernel. To fix this
> > > >     adding a new "which" CPU_WHICH_TIDPID intended to handle both id's.
> > > >
> > > >     Reviewed by:            kib
> > > >     Differential Revision:  https://reviews.freebsd.org/D38209
> > > >     MFC after:              1 week
> > > > ---
> > > >  lib/libc/gen/sched_getaffinity.c |  2 +-
> > > >  lib/libc/gen/sched_setaffinity.c |  2 +-
> > > >  lib/libc/sys/cpuset.2            |  4 +++-
> > > >  sys/kern/kern_cpuset.c           | 42 +++++++++++++++++++++++++++++++++++++++-
> > > >  sys/sys/cpuset.h                 |  1 +
> > > >  5 files changed, 47 insertions(+), 4 deletions(-)
> > >
> > >
> > > Hello,
> > >
> > > This broke math/py-numpy on main and stable/13
> > >
> > > https://pkg-status.freebsd.org/gohan03/data/main-amd64-default-baseline/p52768539d46f_scd3193f640/logs/errors/py39-numpy-1.24.1,1.log
> > > https://pkg-status.freebsd.org/gohan02/data/13stable-amd64-quarterly-baseline/841610d9bfc6/logs/errors/py39-numpy-1.23.5_1,1.log
> > >
> > > File "/wrkdirs/usr/ports/math/py-numpy/work-py39/numpy-1.23.5/numpy/distutils/misc_util.py",
> > > line 93, in get_num_build_jobs
> > >     cpu_count = len(os.sched_getaffinity(0))
> > > OverflowError: could not allocate a large enough CPU set
> >
> > hi, Antoine,
> >
> > on stable/13
> > dchagin@heemeyer:~ % uname -a
> > FreeBSD heemeyer.club 13.2-STABLE FreeBSD 13.2-STABLE #268 stable/13-n254471-51485f81b01a-dirty: Sat Feb 11 12:16:21 MSK 2023     dchagin@heemeyer.club:/usr/home/dchagin/tmp/obj/usr/home/dchagin/freebsd/amd64.amd64/sys/YOY amd64
> > dchagin@heemeyer:~ % python3.9
> > Python 3.9.16 (main, Dec 18 2022, 01:15:32)
> > [Clang 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a on freebsd13
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> import os
> > >>> len(os.sched_getaffinity(0))
> > 1
> > >>> os.sched_getaffinity(0)
> > {0}
> > >>> exit()
> > dchagin@heemeyer:~ %
> >
> > on current:
> > dchagin@mordor:~/freebsd % uname -a
> > FreeBSD mordor.heemeyer.club 14.0-CURRENT FreeBSD 14.0-CURRENT #458 futex-n260850-52456667491d-dirty: Sat Feb 11 14:24:46 MSK 2023     dchagin@mordor.heemeyer.club:/home/dchagin/tmp/obj/home/dchagin/freebsd/amd64.amd64/sys/YOY amd64
> > dchagin@mordor:~/freebsd % python3.9
> > Python 3.9.16 (main, Dec 18 2022, 01:24:21)
> > [Clang 14.0.5 (https://github.com/llvm/llvm-project.git llvmorg-14.0.5-0-gc1238 on freebsd14
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> import os
> > >>> len(os.sched_getaffinity(0))
> > 16
> > >>> os.sched_getaffinity(0)
> > {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}
> > >>> exit()
> > dchagin@mordor:~/freebsd %
> >
> >
> > could you please show out of os.sched_getaffinity(0) ?
>
> and os.cpu_count()
> also sysctl kern.smp

Here is what I have in the head jail:

>>> len(os.sched_getaffinity(0))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: could not allocate a large enough CPU set
>>> os.sched_getaffinity(0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: could not allocate a large enough CPU set
>>> os.cpu_count()
64
>>>
root@mainamd64PR269425-default:~ # sysctl kern.smp
kern.smp.forward_signal_enabled: 1
kern.smp.topology: 0
kern.smp.cores: 32
kern.smp.threads_per_core: 2
kern.smp.cpus: 64
kern.smp.disabled: 0
kern.smp.active: 1
kern.smp.maxcpus: 256
kern.smp.maxid: 63

Antoine