Possible Threading problem with -CURRENT / MySQL?

[Robert Watson]

On Thu, 17 Jun 2004, Brad Knowles wrote:

> At 1:31 AM -0400 2004-06-17, Robert Watson wrote:
> 
> >  - Removing Giant from UNIX domain sockets
> 
> 	Out of curiosity, is this something that could be relatively
> safely done in general?  Any ideas on what the plan is for doing this as
> the default on -CURRENT? 

I'm in the process of merging the necessary support to CVS, but took a
break for a day or two to run some benchmarks and wait for some reviews of
specific pieces of it.  I'll restart the merge process this evening.  You
can find more more information at:

    http://www.watson.org/~robert/freebsd/netperf/

> >  - Disabling HTT
> >  - Using ADAPTIVE_MUTEXES
> 
> 	These both sound like typical improvements, based on what I've
> seen on this list.  Any ideas on when they might become the default?

There's an interaction with HTT that prevents the OS from disabling HTT
when running SCHED_ULE.  The benefits and costs of HTT vary based on load;
with this particular benchmark, HTT hurts quite a bit.

Regarding ADAPTIVE_MUTEXES -- I think we still need more information.  I'm
convinced that on high-contention local IPC workloads, ADAPTIVE_MUTEXES is
a clear win across all other variables -- regardless of scheduler, use of
HTT, fine-grained locking, etc.  However, I haven't benchmarked more
userspace-intensive workloads, and that's where you'd expect it might
hurt.  ADAPTIVE_MUTEXES should (may?) hurt "less" for high user loads when
contention spots are brief, such as in a more fine-grained locking
scenario rather than with a Giant lock.  After I finish the netperf merge,
I'll be doing a set of detailed performance measurements and optimizations
and we'll see if that notion plays out or not.  It could also well be that
ADAPTIVE_MUTEXES are simply a win.  There are also two proposed
modifications to ADAPTIVE_MUTEXES, one by Bosko Milekic and the other by
Alfred Perlstein, and we'll want to look at them as well.

> >  - Running with SCHED_4BSD instead of SCHED_ULE
> 
> 	This is the only one that really concerns me.  This shows that
> we clearly need more work on ULE.  Is there one particular thing that we
> seem to be tripping up on, or is it a multitude of things? 

It's not immediately clear; this benchmark is fairly thread-centric, so it
may just be that ULE is doing a bad job of sharing a quantum among
multiple threads in the same process.  Or, we could be looking at a
load-balancing issue or other issue.  For this benchmark, which involved
lots of smaller IPCs across processors, ULE hurt quite a bit compared to
4BSD.  On UP, the results were identical.  Since this benchmark is a very
specific and narrow workload (yet important), I think we don't yet have
enough information to make a decision as to whether to keep refining ULE
for 5.3, or switch back to 4BSD for 5.3.  Presumably the largest single
concern is actually whether Jeff or others will have time to work on ULE.y

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert at fledge.watson.org      Senior Research Scientist, McAfee Research