Confused about HyperThreading and Performance

[Daniel Ellard]

Can someone point me at some non-marketing documentation about
hyperthreading on the latest Intel chips?  I'm seeing some strange
performance measurements and I would like to figure out what they
mean.

My systems have the following CPU:

CPU:  Intel(R) Xeon(TM) CPU 1.80GHz (1799.81-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf27 Stepping = 7
	  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>

They are running FreeBSD 4.8-RELEASE-p13.

I'm running an application that pumps a lot of data quickly over a
gigabit NIC (an Alteon AceNIC 1000baseT Gigabit Ethernet, using the ti
driver).  The network uses normal ethernet frames (MTU=1500) instead
of jumbo frames, so there are lots of interrupts.  In this case, the
system is mostly reading data from the network, in case that makes a
difference.  The application is multi-threaded, using pthreads.

When I run a kernel using the default configuration (no SMP, no APIC),
the application takes an average of 8.45 seconds (wall-clock time) to
run, and this number is consistent from run to run.  When I run a
kernel built with either SMP or SMP+APIC, it takes an average of 13.25
seconds, and this number is also quite consistent.  (It's not the
Alteon; using the Intel Pro/1000 XT server adapter with the em driver
gives a similar difference, although the intel NIC is slower.)

Interestingly, the CPU utilization for the default kernel is about
75-80%, while for the APIC kernel it is over 95%.  My guess would be
that something is spinning on a lock in the APIC kernel, but that's
just a hunch.

My questions:

1.  Any idea why the performance plummets from 8.45 seconds to 13.25?

2.  Is the situation any better in FreeBSD 5.1 (or 4.9)?  I know in
	4.9 APIC is turned on by default, but if this is the result
	than I won't be upgrading to 4.9 any time soon!

Thanks,
	-Dan