ULE steal_idle questions

[Bruce Evans]

On Sat, 26 Aug 2017, Ian Lepore wrote:

> On Sat, 2017-08-26 at 11:12 -0700, Rodney W. Grimes wrote:
>>>
>>> On Fri, 25 Aug 2017, Don Lewis wrote:

>>> [... context mostly lost to mangling of spaces to \xa0's]

>>> on the newer system.\xa0\xa0SCHED_ULE still has bugfeatures which tend to

Oops, I meant SCHED_4BSD.

>>> help large builds by reducing context switching, e.g., by bogusly
>>> clamping all CPU-bound threads to nearly maximal priority.
>> That last bugfeature is probably what makes current systems
>> interactive performance tank rather badly when under heavy
>> loads.\xa0\xa0Would it be hard to fix?

I fix it in some of my versions of SCHED_4BSD.  It rarely matters.
I even turn off PREEMPTION and IPI_PREEMPTION on SMP systems to
favour large builds with fewer context switches and don't notice
interactivity problems.  This depends on the shell not running the
build and not starting to many CPU hogs so that it stays at numerically
low priority.

> I would second that sentiment... as time goes on, heavily loaded
> systems seem to become less and less interactive-friendly. \xa0Also,
> running the heavy-load jobs such as builds with nice, even -n 20,
> doesn't seem to make any noticible difference in terms of making un-
> nice'd processes more responsive (not sure there's any relationship in
> the underlying causes of that, though).

niceness is quite broken in both SCHED_4BSD and SCHED_ULE.  It was
partly fixed in SCHED_4BSD in ~1999, but re-broken soon after (it
is difficult to map a large dynamic range of CPU usage counts (estcpu)
into the user priority range.  The niceness sub-range wants to more
than it was (81), but even 81 didn't fit and caused bugs.  Fixes
reduced it to 41 where it barely does anything.  ULE intentionally
copied some bugs from this (like ensuring that nice -20 processes
never run in competetion with nice --20 processes).

This is fixed in SCHED_4BSD in some of my versions, but I only use
nice to test the fix.  The implementation uses virtual ticks, with
nice'd processes being charged more.  This is almost perfectly fair,
with the relative CPU allocation following a table, but Linux in 2004
somehow does better (except for no way to change the policy) using
an apparently much simpler algorithm.

Bruce