Tuning the scheduler? Desktop with a CPU-intensive task becomes rapidly unusable.

[Jeremy Chadwick]

On Mon, Sep 06, 2010 at 03:05:53PM -0400, jhell wrote:
> On 09/03/2010 20:14, David Xu wrote:
> > jan.grant at bristol.ac.uk wrote:
> >> On Thu, 2 Sep 2010, Andriy Gapon wrote:
> >> 
> >> 
> >>> on 02/09/2010 12:08 jan.grant at bristol.ac.uk said the following:
> >>> 
> >>>> On Wed, 1 Sep 2010, Ivan Voras wrote:
> >>>> 
> >>>> 
> >>>>> On 09/01/10 15:08, jan.grant at bristol.ac.uk wrote:
> >>>>> 
> >>>>>> I'm running -STABLE with a kde-derived desktop. This setup
> >>>>>> (which is pretty standard) is providing abysmal interactive
> >>>>>> performance on an eight-core machine whenever I try to do
> >>>>>> anything CPU-intensive (such as building a port).
> >>>>>> 
> >>>>>> Basically, trying to build anything from ports rapidly
> >>>>>> renders everything else so "non-interactive" in the eyes of
> >>>>>> the scheduler that, for instance, switching between virtual
> >>>>>> desktops (I have six of them in reasonably frequent use)
> >>>>>> takes about a minute of painful waiting on redraws to 
> >>>>>> complete.
> >>>>>> 
> >>>>> Are you sure this is about the scheduler or maybe bad X11
> >>>>> drivers?
> >>>>> 
> >>>> Not 100%, but mostly convinced; I've just started looking at
> >>>> this. It's my first stab at what might be going on. X11
> >>>> performance is usually pretty snappy. There's no paging
> >>>> pressure at all.
> >>>> 
> >>> From my experience: 1. system with Athlon II X2 250 CPU and
> >>> onboard AMD graphics - no issues with interaction between
> >>> buildworld and GUI with all KDE4 effects enabled (OpenGL). 2.
> >>> system with comparable Core2 Duo CPU and onboard Intel graphics 
> >>> (G33) - enabling OpenGL desktop effects in KDE4 leads to the
> >>> consequences like what you describe.  With all GUI bells and
> >>> whistles disabled the system behaves quite like the AMD system.
> >>> 
> >> 
> >> All desktop effects are disabled. The graphics are from an nVidia 
> >> GeForce 8500 GT (G86) with the X.org driver. (It's not _just_
> >> desktop behaviour that's affected, though: the box runs a number of
> >> small headless [interactive] server processes which also appear to
> >> get rapidly starved of CPU time.)
> >> 
> >> The behaviour isn't visible with the 4bsd scheduler; "stuff"
> >> generally remains snappy and responsive.
> >> 
> >> I'll keep poking around and see if I can get to the bottom of it.
> >> 
> >> 
> >> 
> >> 
> > I think sysctl kern.sched.preempt_thresh is too low, default is only
> > 64. I always tune it up to 200 on  my desktop machine which is
> > running gnome and other GUI applications, for a heavy GUI deskkop, I
> > would tune it up to 224 to get better result.
> > 
> 
> For reference how did you arrive at 224 for a result ?

Can someone explain exactly what this sysctl does?  The description is
only useful if you have familiarity with the scheduler internals:

$ sysctl -d kern.sched.preempt_thresh
kern.sched.preempt_thresh: Min priority for preemption, lower priorities have greater precedence

The source code doesn't really explain it either -- but I will point out
that there's a change in the default value based on an option called
FULL_PREEMPTION:

src/sys/kern/sched_ule.c
 192 #ifdef PREEMPTION
 193 #ifdef FULL_PREEMPTION
 194 static int preempt_thresh = PRI_MAX_IDLE;
 195 #else
 196 static int preempt_thresh = PRI_MIN_KERN;
 197 #endif
 198 #else
 199 static int preempt_thresh = 0;
 200 #endif

src/sys/sys/priority.h
  81 #define PRI_MAX                 (255)           /* Lowest priority. */
  97 #define PRI_MIN_KERN            (64)
 ...
 121 #define PRI_MAX_IDLE            (PRI_MAX)

Secondly, this tunable isn't mentioned in any man page, so I'm not sure
how users would even come across it:

$ zgrep -r preempt_thresh /usr/share/man
$

Thirdly, does this sysctl only exist if "options PREEMPTION" is
included in one's kernel config?  (I did not dig through kernel source
thoroughly)

-- 
| Jeremy Chadwick                                   jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                  Mountain View, CA, USA |
| Making life hard for others since 1977.              PGP: 4BD6C0CB |