Call for thread testers

[Daniel Eischen]

On Thu, 28 Aug 2003, David Xu wrote:

> On Thursday 28 August 2003 06:07, Daniel Eischen wrote:
> > [ redirected to @threads ]
> > > David and I have a couple of other tests which we use to
> > > measure performance of our threads libraries.
> >
> > One of these tests is crew, which is part of Dave Butenhof's
> > threads tests (http://people.freebsd.org/~davidxu/crew/programs.tar.z)
> >
> >   Usage: crew <string> <directory>
> >
> > crew searches recursively for the given string in the directory.
> > David and I use it as follows:
> >
> >   $ crew node /usr/src/sys
> >
> > By default, it creates 4 worker threads to seach for the string.
> > Here are the results for all thread libraries; they are somewhat
> > surprising.  The test was run a couple of times to fill up cache
> > and those timing results were omitted.
> >
> >                        Run 1       Run 2       Run 3
> >   ------------------------------------------------------
> >   libc_r       real    0m48.451s   0m48.835s   0m50.905s
> >                user    0m2.694s    0m2.745s    0m2.818s
> >                sys     0m8.996s    0m8.931s    0m8.863s
> >   ------------------------------------------------------
> >   libthr       real    1m2.553s    1m52.579s   1m50.306s
> >                user    0m3.423s    0m4.561s    0m4.397s
> >                sys     0m11.842s   0m27.099s   0m27.300s
> >   ------------------------------------------------------
> >   libkse(M:N)  real    0m49.175s   0m49.910s   0m51.033s
> >                user    0m2.840s    0m2.836s    0m2.880s
> >                sys     0m8.857s    0m8.857s    0m8.792s
> >   ------------------------------------------------------
> >   libkse(1:1)  real    0m20.240s   0m19.969s   0m19.981s
> >                user    0m2.741s    0m2.783s    0m2.858s
> >                sys     0m8.877s    0m8.832s    0m8.768s
> >   ------------------------------------------------------
> >
> > All the above tests were run on a single CPU 850MHz PIII
> > with 256MB RAM (Dell Lattitude C400).
> >
> > There seems to be something wrong with libthr; the times
> > vary too much (by almost 100%).  libkse in M:N mode seems
> > to be on par with libc_r.  But the real surprise is libkse
> > built in 1:1 mode (add -DSYSTEM_SCOPE_ONLY to CFLAGS when
> > building libpthread).
> 
> Er, strange result, libkse M:N and libkse 1:1 should be identical 
> in this test, I had always run this test, and their performance
> are always close enough, but 1:1 was always a bit slow than M:N,
> all tests are done under 4BSD scheduler, under ULE scheduler, 
> SA process has performance problem, M:N is slower under ULE than 1:1.

Yes, perhaps my kernel was a bit out of date.  I also had
forgotten I had libc_r mapped to libkse with libmap.conf,
so the libc_r tests were actually using libkse!  I re-ran
the tests on a different box, single PIII 800MHz, 512MB RAM.
They look better, although libthr still doesn't give consistent
results.

                       Run 1         Run 2          Run 3
  -----------------------------------------------------------
  libc_r       real    0m13.739s     0m13.739s      0m13.882s
               user    0m3.330s      0m3.302s       0m3.394s
               sys     0m9.858s      0m9.893s       0m9.820s
  -----------------------------------------------------------
  libkse(M:N)  real    0m11.977s     0m12.199s      0m12.097s
               user    0m3.248s      0m3.081s       0m2.857s
               sys     0m8.190s      0m8.517s       0m8.575s
  -----------------------------------------------------------
  libkse(1:1)  real    0m11.972s     0m12.044s      0m12.035s
               user    0m3.198s      0m2.980s       0m3.183s
               sys     0m8.244s      0m8.480s       0m8.282s
  -----------------------------------------------------------
  libthr       real    0m34.180s     0m16.193s      0m34.119s
               user    0m5.075s      0m3.874s       0m5.255s
               sys     0m28.286s     0m11.626s      0m28.038s
  -----------------------------------------------------------

libkse(1:1) and libkse(M:N) are about equal, and slightly
better than libc_r.  I can't explain libthr results.

-- 
Dan Eischen