SCHED_ULE should not be the default

Tue Dec 13 12:47:10 UTC 2011

On Tue, Dec 13, 2011 at 12:13:42PM +0100, O. Hartmann wrote:
> On 12/12/11 16:13, Vincent Hoffman wrote:
> > 
> > On 12/12/2011 13:47, O. Hartmann wrote:
> > 
> >>> Not fully right, boinc defaults to run on idprio 31 so this isn't an
> >>> issue. And yes, there are cases where SCHED_ULE shows much better
> >>> performance then SCHED_4BSD. [...]
> > 
> >> Do we have any proof at hand for such cases where SCHED_ULE performs
> >> much better than SCHED_4BSD? Whenever the subject comes up, it is
> >> mentioned, that SCHED_ULE has better performance on boxes with a ncpu >
> >> 2. But in the end I see here contradictionary statements. People
> >> complain about poor performance (especially in scientific environments),
> >> and other give contra not being the case.
> > It all a little old now but some if the stuff in
> > http://people.freebsd.org/~kris/scaling/
> > covers improvements that were seen.
> > 
> > http://jeffr-tech.livejournal.com/5705.html
> > shows a little too, reading though Jeffs blog is worth it as it has some
> > interesting stuff on SHED_ULE.
> > 
> > I thought there were some more benchmarks floating round but cant find
> > any with a quick google.
> > 
> > 
> > Vince
> > 
> > 
> 
> Interesting, there seems to be a much more performant scheduler in 7.0,
> called SCHED_SMP. I have some faint recalls on that ... where is this
> beast gone?

Boy I sure hope I remember this right.  I strongly urge others to
correct me where I'm wrong; thanks in advance!

The classic scheduler, SCHED_4BSD, was implemented back before there was
oxygen.  sched_4bsd(4) mentions this.  No need to discuss it.

Jeff Robertson began working on the "first-generation ULE scheduler"
during the days of FreeBSD 5.x (I believe 5.1), and a paper on it was
presented at USENIX circa 2003:
http://www.usenix.org/event/bsdcon03/tech/full_papers/roberson/roberson.pdf

Over the following years, Jeff (and others I assume -- maybe folks like
George Neville-Neil and/or Kirk McKusick?) adjusted and tinkered with
some of the semantics and models/methods.  If I remember right, some of
these quirks/fixes were committed.  All of this was happening under the
scheduler that was then called SCHED_ULE, but it was "ULE 1.0" for lack
of better terminology.

This scheduler did not perform well, if I remember right, and Jeff was
quite honest about that.  From this point forward, Jeff began idealising
and working on a scheduler which he called SCHED_SMP -- think of it as
"ULE 2.0", again, for lack of better terminology.  It was different than
the existing SCHED_ULE scheduler, hence a different name.  Jeff blogged
about this in early 2007, using exactly that term ("ULE 2.0"):
http://jeffr-tech.livejournal.com/3729.html

In mid-2007, prior to FreeBSD 7.0-RELEASE, Jeff announced that
effectively he wanted to make SCHED_ULE do what SCHED_SMP did, and
provided a patch to SCHED_ULE to accomplish just that:
http://unix.derkeiler.com/Mailing-Lists/FreeBSD/current/2007-07/msg00755.html

Full thread is here (beware -- many replies):
http://unix.derkeiler.com/Mailing-Lists/FreeBSD/current/2007-07/threads.html#00755

The patch mentioned above was merged into HEAD on 2007/07/19.
http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/sched_ule.c#rev1.202

So in effect, as of 2007/07/19, SCHED_ULE became SCHED_SMP.

FreeBSD 7.0-RELEASE was released on 2008/02/27, and the above
commit/changes were available at that time as well (meaning: RELENG_7
and RELENG_7_0 at that moment in time should have included the patch
from the above paragraph).

The document released by Kris Kenneway hinted at those changes and
performance improvements:
http://people.freebsd.org/~kris/scaling/7.0%20Preview.pdf

Keep in mind, however, that at that time kernel configuration files
(GENERIC, etc.) still defaulted to SCHED_4BSD.

The default scheduler in kernel config files (GENERIC, etc.) for i386
and amd64 (not sure about others) was changed in 2007/10/19:
http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/i386/conf/GENERIC#rev1.475
http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/amd64/conf/GENERIC#rev1.485

This was done *prior* to FreeBSD 7.1-RELEASE.  So, it first became
available as the default scheduler "for the masses" when 7.1-RELEASE
came out on 2009/01/05.

"All of the answers", in a roundabout and non-user-friendly way, are
available by examining the commit history for src/sys/kern/sched_ule.c.
It's hard to follow especially given that you have to consider all
the releases/branchpoints that took place over time, but:
http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/sched_ule.c

Are we having fun yet?  :-)

-- 
| Jeremy Chadwick                                jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                   Mountain View, CA, US |
| Making life hard for others since 1977.               PGP 4BD6C0CB |