machdep.cpu_idle_hlt and SMP perf?

[John Baldwin]

On Monday 06 February 2006 14:46, Andrew Gallatin wrote:
> Andre Oppermann writes:
>  > Andrew Gallatin wrote:
>  > > Why dooes machdep.cpu_idle_hlt=1 drop my 10GbE network rx
>  > > performance by a considerable amount (7.5Gbs -> 5.5Gbs)?
>
> <...>
>
>  > This may be the same problem OpenBSD has fixed last year in the handling
>  > of the idle loop.  From the kerneltrap posting:
>
> <....>
>
>  > First commit message:
>  >   http://marc.theaimsgroup.com/?l=openbsd-cvs&m=111692513727274&w=2
>  >
>  > The MFC with all changes in one commit message:
>  >   http://marc.theaimsgroup.com/?l=openbsd-cvs&m=111859519015510&w=2
>
> The bug they fixes was missing interrupts by both calling APM's idle
> routine, which may hlt, and hlt'ing in the idle loop itself.  Since I
> have no idea what acpi is doing, I got excited about this.
>
> Alas, it seems like this isn't it.  I pointed cpu_idle_hook back to
> cpu_idle_default and away from acpi_cpu_idle, but that made no
> difference.

You may be seeing problems because it might simply take a while for the CPU to 
wake up from HLT when an interrupt comes in.  The 4BSD scheduler tries to do 
IPIs to wakeup any sleeping CPUs when it schedules a new thread, but that 
would add higher latency for ithreads than just preempting directly to the 
ithread.  Oh, you have to turn that on, it's off by default 
(kern.sched.ipiwakeup.enabled=1).

-- 
John Baldwin <jhb at FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org