SCHEDULE and high load situations

[Robert Watson]

On Wed, 11 Aug 2004, Martin Blapp wrote:

> During load tests with both SCHED4BSD and SCHEDULE I found that the
> latest SCHEDULE version cannot have a load over 2-3. I stress tested the
> server really a lot, but about 500 processes did sleep and did not got
> scheduled and only 2 of 500 were run and got about 50% of CPU time. 
> 
> Is this a known problem ? 

I've found that for throughput oriented workloads, 4BSD substantially
outperforms ULE, but I haven't tried it with Jeff's latest set of patches
(committed a day or two ago).  You don't mention if your box is SMP, btw
-- I've noticed some load balancing problems with ULE previously, but
haven't checked if they were resolved.  Anecdotal opinion seems generally
to be that interactivity is observably better with ULE than 4BSD, but that
4BSD appears to do a better job under load.

This weekend I'll be rerunning a number of benchmarks across a broad range
of variables (scheduler, smp support, htt, various locking models,
different netisr models, threading libraries, etc) to evaluate progress
over the last month, and I'll see how things look with ULE.

FYI, when I started benchmarking MySQL a couple of months ago, I was
seeing about 2800 transactions/sec on a benchmark on my dual P4 with
(GENERIC - DEBUGGING).  Now I'm seeing about 7100-7200 transactions/sec. 
I've merged pretty much all of the changes necessary to see that
transition except moving to 4BSD by default, disabling HTT by default, and
enabling Giant-free networking by default.  UP has also improved, but only
be a relatively small fraction, so I've been shifting attention to UP from
SMP.  Some of the wins on SMP have been from moving to adaptive mutexes by
default (most recently, for Giant on i386); others from improved fine
grain locking in VM and networking, and general optimization of
synchronization primitives, scheduling, wakeups/locking, etc.

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert at fledge.watson.org      Principal Research Scientist, McAfee Research