To SMP or not to SMP

[Barney Cordoba]

--- On Wed, 1/9/13, Barney Cordoba <barney_cordoba at yahoo.com> wrote:

> From: Barney Cordoba <barney_cordoba at yahoo.com>
> Subject: Re: To SMP or not to SMP
> To: "Mark Atkinson" <atkin901 at gmail.com>
> Cc: freebsd-net at freebsd.org
> Date: Wednesday, January 9, 2013, 1:08 PM
> 
> 
> --- On Tue, 1/8/13, Mark Atkinson <atkin901 at gmail.com>
> wrote:
> 
> > From: Mark Atkinson <atkin901 at gmail.com>
> > Subject: Re: To SMP or not to SMP
> > To: freebsd-net at freebsd.org
> > Date: Tuesday, January 8, 2013, 11:29 AM
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> > 
> > On 01/07/2013 18:25, Barney Cordoba wrote:
> > > I have a situation where I have to run 9.1 on an
> old
> > single core
> > > box. Does anyone have a handle on whether it's
> better
> > to build a
> > > non SMP kernel or to just use a standard SMP build
> with
> > just the
> > > one core? Thanks.
> > 
> > You can build a SMP kernel, but you'll get better
> > performance (in my
> > experience) with SCHED_4BSD on single cpu than with
> ULE.
> > 
> 
> I've tested the 2 schedulers on an SMP kernel with 1 core. I
> don't have
> a 1 core system to test with so I'm using an E5520
> with  1 core enabled.
> 
> Bridging a controlled test (curl-loader doing a web-load
> test with 100 
> users that consistently generates 870Mb/s and 77Kpps, I see
> the following:
> 
> top -SH
> 
> ULE:
> 
> idle: 74.85%
> kernel {em1 que} 17.68%
> kernel {em0 que} 5.86%
> httpd: .49%  
> 
> 4BSD:
> 
> idle: 70.95%
> kernel {em1 que} 18.07%
> kernel {em0 que} 4.44%
> httpd: .93%
> 
> Note that the https is a monitor I'm running.
> 
> so it appears that theres 7% of usage missing (all other
> apps show 0%
> usage). 
> 
> If i had to guess just looking at the numbers, it seems that
> 4BSD might 
> do better with the interrupt level stuff, and not as good
> with user 
> level context switching. I think they're close enough to
> stick with ULE
> so I can just use a stock kernel.
> 
> One thing that bothers me is the idle sits at 100% when
> other tasks are 
> registering values under light loads, so it's certainly not
> all that 
> accurate.
> 
> BC

Ok, thanks to J Baldwin's tip I got a NON-SMP kernel running with some
interesting results. Here's all 4 tests:

I've tested the 2 schedulers on an SMP kernel with 1 core. I don't have
a 1 core system to test with so I'm using an E5520 with  1 core enabled.

Bridging a controlled test (curl-loader doing a web-load test with 100 
users that consistently generates 870Mb/s and 77Kpps, I see the following:

top -SH

ULE (SMP):

idle: 74.85%
kernel {em1 que} 17.68%
kernel {em0 que} 5.86%
httpd: .49%  

4BSD (SMP):

idle: 70.95%
kernel {em1 que} 18.07%
kernel {em0 que} 4.44%
httpd: .93%

4BSD (NON-SMP):

idle: 72.95%
kernel {em1 que} 15.04%
kernel {em0 que} 6.10%
httpd: 1.17%

ULE (NON-SMP):

idle: 76.17%
kernel {em1 que} 16.99%
kernel {em0 que} 5.18%
httpd: 1.66%

A kernel with SMP off seems to be a bit more efficient. A better test would
be to have more stuff running, but Im about out of time on this project.

BC