To SMP or not to SMP

[Konstantin Belousov]

On Wed, Jan 09, 2013 at 03:54:43PM -0500, John Baldwin wrote:
> On Wednesday, January 09, 2013 10:00:31 am Barney Cordoba wrote:
> > 
> > --- On Wed, 1/9/13, sthaug at nethelp.no <sthaug at nethelp.no> wrote:
> > 
> > > From: sthaug at nethelp.no <sthaug at nethelp.no>
> > > Subject: Re: To SMP or not to SMP
> > > To: erichsfreebsdlist at alogt.com
> > > Cc: barney_cordoba at yahoo.com, freebsd-net at freebsd.org, 
> jack.vogel at gmail.com, atkin901 at gmail.com
> > > Date: Wednesday, January 9, 2013, 9:32 AM
> > > > > 4BSD runs pretty well with
> > > an SMP kernel. I can test ULE and compare
> > > > > easily. A no SMP kernel is problematic as the igb
> > > driver doesn't seem
> > > > > to work and my onboard NICs are, sadly, igb. 
> > > > > 
> > > > this is bad luck. I know of the kernels as I have had
> > > SMP and single
> > > > CPU machines since 4.x times.
> > > 
> > > I have had igb working with both SMP and non-SMP kernel for
> > > at least a
> > > year or two, 8.x-STABLE. No specific problems.
> > > 
> > > Steinar Haug, Nethelp consulting, sthaug at nethelp.no
> > > 
> > 
> > Maybe a problem with "legacy interrupts" on more modern processors?
> > I'm using an E5520 and while the NIC inits ok, it just doesnt seem to
> > gen interrupts. I can't spend much time debugging it....
> > 
> > I notice that HAMMER kernels use MSI/X interrupts whether SMP is enabled
> > or not, while i386 kernels seem to require APIC. Is there some physical
> > reason for this?
> 
> MSI always requires APIC.  (MSI interrupts on x86 have to be delivered to a 
> local APIC, no way to send them to 8259As.)  You can build an i386 kernel with 
> device apic but without 'options SMP' which is akin to leaving SMP out of an 
> amd64 kernel.
> 
> Removing SMP on x86 changes the following things:
> - Spin mutexes just disable interrupts on the local CPU and don't use any
>   atomic operations at all.  All other lock types work the same.
> - atomic operations don't use the "lock" prefix so are cheaper.  However, the
>   atomic op used for locks (cmpxchg) has an implicit "lock" prefix, so this
>   isn't but so much of a gain.
I do not think that cmpxchg uses implicit lock, and Intel IA32 SDM supports
this view. The absense of the lock prefix makes the instruction decode
faster, and removes the locked bus cycle.

It is xchg which is implicitely locked, but we no longer use it for unlock.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 834 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-net/attachments/20130110/021ca2aa/attachment.sig>