Re: Periodic rant about SCHED_ULE

On Mar 22, 2023, at 15:39, Mark Millard <marklmi@yahoo.com> wrote:

> On Mar 22, 2023, at 13:34, Mark Millard <marklmi@yahoo.com> wrote:
> 
>> On Mar 22, 2023, at 12:40, George Mitchell <george+freebsd@m5p.com> wrote:
>> 
>>> On 3/22/23 15:21, Mark Millard wrote:
>>>> George Mitchell <george+freebsd@m5p.com> wrote on
>>>> Date: Wed, 22 Mar 2023 17:36:39 UTC :
>>>> [...]
>>>>> Here are the very complicated instructions for reproducing the problem:
>>>>> 1. Install and start misc/dnetc from ports.
>>>> Installing is likely easy, as likely would be building
>>>> with default options (if any). I know nothing about
>>>> starting misc/dnetc so that is research. (Possibly
>>>> trivial, although if it has alternatives to control
>>>> then I'd need to match that context too.)
>>> 
>>> service dnetc start
>> 
>> I built and installed misc/dnetc and got a binary
>> blob that clearly was not built in my environment:
>> 
>> # file /usr/local/distributed.net/dnetc
>> /usr/local/distributed.net/dnetc: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), statically linked, for FreeBSD 10.1 (1001515), FreeBSD-style, stripped
>> 
>> Way older FreeBSD vintage than the locally available toolchains
>> would normally build. Some might be cautious about such a thing.
>> 
>> The man page reported that:
>> 
>> QUOTE
>>    If you have never run the client before, it will initiate the menu-driven
>>    configuration. Save and quit when done, the configuration file will be
>>    saved in the same directory as the client. Now, simply restart the
>>    client. From that point on it will use the saved configuration.
>> END QUOTE
>> 
>> I've not seen what the configuration asks about yet.
> 
> I went through the configuration, basically just looking
> at it, other than providing an E-mail address. Then . . .
> 
> $ sudo service dnetc start
> Password:
> Cannot 'start' dnetc. Set dnetc_enable to YES in /etc/rc.conf or use 'onestart' instead of 'start'.
> 
> $ sudo service dnetc onestart
> 
> I just let it run without any extra competing activity, other
> than I had my patched version of top running. It records and
> reports various maximum-observed (MaxObs) figures, here
> the load averages being relevant.
> 
> Top showed that dnetc started 32 processes, one per hardware
> thread. Mostly I saw: 100% nice and 0% idle.
> 
> Letting it run and then looking at the load averages (and
> their matching MaxObs figures) after something like 60+ min
> (not carefully timed: was doing other things) showed:
> 
> load averages:  31.97,  31.88,  31.66 MaxObs:  32.12,  31.97,  31.66
> 
> (Note: The machine had been up for over 2.75 days before
> starting this and had not been building much of anything
> during that time.)
> 
> I've not yet experimented with having other, significant
> competing activity.
> 
>>>>> 2. Run "make buildworld".
>>>> So on the 32 hardware-thread (16 cores) amd64 machine that
>>>> I have access to, the test is to only have buildworld use
>>>> about one hardware thread, no matter what else is going on.
>>>> I never would have guessed that the steps would not involve
>>>> more like -j$(sysctl -n hw.ncpu) (so around -j32 in this
>>>> context). So it is good that you provided your note or
>>>> I'd not know if I'd done similarly or not when trying such.
>>>> [Note: -j1 and lack of -j are not strictly equivalent in
>>>> how make operates. As I remember, the distinction makes
>>>> a notable difference in the number of subprocesses created
>>>> directly by make (one per action "line" vs. one for the
>>>> whole block?). So even using -j1 might make a difference
>>>> vs. what you specified. I'd have to test to see.]
>>> 
>>> I am literally running "make buildworld" with no additional options.
>> 
>> So required for repeating your results, but likely making
>> such results not be interesting relative to how I normally
>> deal with buildworld buildkernel and the likel, no matter
>> if there is other activity in an overlapping time frame or
>> not: my time preferences are too strong to wait for a single
>> hardware thread to do my normal builds, even with no
>> competing activity on the builder.
>> 
>>>>> Standard out conveniently reports how long it took (wall clock).
>>>> But nothing in your instructions indicate about how
>>>> to get an idea much progress dnetc made during the
>>>> various tests? [...]
>>> 
>>> Honestly, I've never worried about this part.  But dnetc logs its
>>> progress in /usr/local/distributed.net/dnetc.txt, though not in terms
>>> that are easy to relate to real-world progress.  Oddly, when I run
>>> "make buildworld," I'm primarily interested in getting the world built.
>>> Perhaps others feel differently.
>> 
>> Off topic for the specifics of the actual benchmark
>> that you run:
>> 
>> Then why not use of -jN ? In my context, any buildworld
>> using -j1 or no -j at all takes a huge amount of time
>> longer than letting it use all the hardware threads (or
>> so). (I've avoided having any I/O bound contexts for
>> such.) It does not take additional load on the system
>> for that to be true --including on the 4-core small arm
>> boards when I happen to buildworld on such (rare).
>> 
>> 
>>>> [...]
>>>> FYI: I've never built with and run the alternate
>>>> scheduler so if there is any appropriate background
>>>> for that that would not be obvious on finding basic
>>>> instructions, it would be appropriate to provide
>>>> such notes.
>>>> [...]
>>> 
>>> You have to build a new kernel, using a config file in which you have
>>> replaced "options SCHED_ULE" with "options SCHED_4BSD".     -- George
>> 
>> Thanks for the notes.
>> 
>> I've not decided if I'll do anything with the binary
>> blob or not.
> 

FYI:

It is not your specific experiment, but I started my
"extra load" experimenst with . . .

I started a -j32 buildworld buildkernel with dnetc still
running. I'm generally seeing around 55% Active and 42%
nice, < 2% system (it was building libllvm at this point).
At that time:

load averages:  64.41,  60.52,  49.81 MaxObs:  64.47,  60.52,  49.81


===
Mark Millard
marklmi at yahoo.com