"cpuset -n prefer:?" --what values for "?" are supposed to be allowed? (only 1 is, despite two numa domains)

[Mark Johnston]

On Wed, Sep 11, 2019 at 11:14:42AM -0700, Mark Millard wrote:
> 
> 
> On 2019-Sep-11, at 10:11, Mark Millard <marklmi at yahoo.com> wrote:
> 
> 
> 
> > On 2019-Sep-11, at 08:15, Mark Johnston <markj at freebsd.org> wrote:
> > 
> >> On Wed, Sep 11, 2019 at 07:57:26AM -0700, Mark Millard wrote:
> >>> 
> >>> 
> >>> On 2019-Sep-11, at 07:31, Mark Johnston <markj at freebsd.org> wrote:
> >>> 
> >>>> On Tue, Sep 10, 2019 at 10:58:05PM -0700, Mark Millard wrote:
> >>>>> In a context with:
> >>>>> 
> >>>>> # cpuset -g
> >>>>> pid -1 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27
> >>>>> pid -1 domain policy: first-touch mask: 0, 1
> >>>>> 
> >>>>> I get:
> >>>>> 
> >>>>> # cpuset -l0 -n prefer:0 COMMAND
> >>>>> cpuset: setdomain: Invalid argument
> >>>>> 
> >>>>> # cpuset -l0 -n prefer:2 COMMAND
> >>>>> cpuset: setdomain: Invalid argument
> >>>>> 
> >>>>> But one prefer:? value does allow the COMMAND
> >>>>> to run:
> >>>>> 
> >>>>> # cpuset -l0 -n prefer:1 COMMAND
> >>>>> 
> >>>>> This seem odd to me. Am I missing something?
> >>>>> 
> >>>>> For reference: I'm using a ThreadRipper 1950X
> >>>>> with a head -r351227 based context for this
> >>>>> activity. The above happens to have been run
> >>>>> in a Windows 10 Pro HyperV session, instead
> >>>>> of in a native-boot of the same media. (A
> >>>>> native-boot would have had 32 CPUs.)
> >>>> 
> >>>> Can you please show the output of "sysctl vm.phys_segs" from this
> >>>> setup?
> >>> 
> >>> Sure:
> >> 
> >> I was wondering if you had only one domain populated, but it seems not
> >> to be the case.  Could you try updating to r351672 or later and see if
> >> the behaviour persists?
> > 
> > It may be a bit before I do that.
> > 
> > FYI: I had set MAXMEMDOM to match the number of
> > actual domains for the context:
> > 
> > /usr/src/sys/amd64/conf/GENERIC-DBG:options     MAXMEMDOM=2
> > /usr/src/sys/amd64/conf/GENERIC-NODBG:options   MAXMEMDOM=2
> > 
> > (These kernel configuration files include GENERIC.)

Ok, that helps.  I believe you are hitting a bug that will be fixed by
r351672 and a couple of preceding commits to the same area.

> Not that the below is the problem that I reported, but
> cpuset_modify_domain has an oddity. In the below, note
> the "root->" use followed by the "root &&" test: the
> root-> use would have failed first. Should the && be
> "dset &&" instead? Should "root &&" just be removed for
> being redundant?

Good catch.  I believe cpusets are not allowed to have a NULL domain
set, so dset should never be NULL either.