ULE steal_idle questions

[Konstantin Belousov]

On Sat, Aug 26, 2017 at 11:29:29AM -0700, Don Lewis wrote:
> I actually haven't noticed that problem on my package build boxes.  I've
> experienced decent interactive performance even when the load average is
> in the 60 to 80 range.  I also have poudriere configured to use tmpfs
> and the only issue I run into is when it starts getting heavily into
> swap (like 20G) and I leave my session idle for a while, which lets my
> shell and sshd get swapped out.  Then it takes them a while to wake up
> again.  Once they are paged in, then things feel snappy again.  This is
> remote access, so I can't comment on what X11 feels like.

I believe what people complain about is the following scenario:
they have some interactive long living process, say firefox or mplayer.
The process' threads consume CPU cycles, so the ULE interactivity
detection logic actually classifies the threads as non-interactive.

This is not much problematic until a parallel build starts where
toolchain processes are typically short-lived.  This makes them
classified as interactive, and their dynamic priority are lower than the
priority of long-lived threads which are interactive by user perception.

I did not analyzed the KTR dumps but this explanation more or less
coincides with the system slugginess when attempt to use mplayer while
heavily oversubscribed build (e.g. make -j 10 on 4 cores x 2 SMT
machine) is started.