Call for thread testers

[Daniel Eischen]

[ redirected to @threads ]
On Wed, 27 Aug 2003, Daniel Eischen wrote:

> On Wed, 27 Aug 2003, Scott Long wrote:
> 
> > All,
> > 
> > This is kind of an unconventional call for help.  As we approach the
> > release of 5.2, we'd really like to show off the performance and
> > stability of our new threading packages.  So, I'm looking for people
> > to volunteer to go out and put some of the thread-capable enterprise
> > and desktop packages to the test.  Packages that I would most like
> > to see are:
> > 
> > Apache 2
> > BIND 9
> > OpenOffice
> > Mozilla
> > KDE desktop + Konq
> > Java 1.3/1.4 + Tomcat/etc
> > MySQL
> > Others?
> 
> As to OpenOffice, Mozilla, and KDE/Konqueror, David and I have
> been using them all with libkse with no noticable difference.
> I don't know how you'd judge performance with these, since
> it is more "how it feels".
> 
> You shouldn't use jdk13 with libkse or libthr since it relies
> on the internals of the thread library (libc_r).  It is only useful
> to benchmark jdk14 which, in theory, should work with all our
> native thread libraries.
> 
> > The questions that I would like to have answered are:
> > 
> > Does it take advantage of pthreads
> > Does it work with libkse and libthr
> > Does it operate correctly
> > Does it perform better/worse than with libc_r
> > Does it perform better/worse than under FreeBSD 4.x
> 
> One thing I have noticed about Mozilla, is that it does
> create both scope process and scope system threads, which
> was kind of nice to see.
> 
> The ACE library is threaded and includes a set of tests;
> some of which show performance:
> 
>   http://www.cs.wustl.edu/~schmidt/ACE.html
> 
> David and I have a couple of other tests which we use to
> measure performance of our threads libraries.

One of these tests is crew, which is part of Dave Butenhof's
threads tests (http://people.freebsd.org/~davidxu/crew/programs.tar.z)

  Usage: crew <string> <directory>

crew searches recursively for the given string in the directory.
David and I use it as follows:

  $ crew node /usr/src/sys

By default, it creates 4 worker threads to seach for the string.
Here are the results for all thread libraries; they are somewhat
surprising.  The test was run a couple of times to fill up cache
and those timing results were omitted.

                       Run 1       Run 2       Run 3
  ------------------------------------------------------
  libc_r       real    0m48.451s   0m48.835s   0m50.905s
               user    0m2.694s    0m2.745s    0m2.818s
               sys     0m8.996s    0m8.931s    0m8.863s
  ------------------------------------------------------
  libthr       real    1m2.553s    1m52.579s   1m50.306s
               user    0m3.423s    0m4.561s    0m4.397s
               sys     0m11.842s   0m27.099s   0m27.300s
  ------------------------------------------------------
  libkse(M:N)  real    0m49.175s   0m49.910s   0m51.033s
               user    0m2.840s    0m2.836s    0m2.880s
               sys     0m8.857s    0m8.857s    0m8.792s
  ------------------------------------------------------
  libkse(1:1)  real    0m20.240s   0m19.969s   0m19.981s
               user    0m2.741s    0m2.783s    0m2.858s
               sys     0m8.877s    0m8.832s    0m8.768s
  ------------------------------------------------------

All the above tests were run on a single CPU 850MHz PIII
with 256MB RAM (Dell Lattitude C400).

There seems to be something wrong with libthr; the times
vary too much (by almost 100%).  libkse in M:N mode seems
to be on par with libc_r.  But the real surprise is libkse
built in 1:1 mode (add -DSYSTEM_SCOPE_ONLY to CFLAGS when
building libpthread).

-- 
Dan Eischen