machdep.cpu_idle_hlt and SMP perf?

[John Baldwin]

On Monday 06 February 2006 17:37, Andrew Gallatin wrote:
> John Baldwin writes:
>  > On Monday 06 February 2006 14:46, Andrew Gallatin wrote:
>  > > Andre Oppermann writes:
>  > >  > Andrew Gallatin wrote:
>  > >  > > Why dooes machdep.cpu_idle_hlt=1 drop my 10GbE network rx
>  > >  > > performance by a considerable amount (7.5Gbs -> 5.5Gbs)?
>  > >
>  > > <...>
>  > >
>  > >  > This may be the same problem OpenBSD has fixed last year in the
>  > >  > handling of the idle loop.  From the kerneltrap posting:
>  > >
>  > > <....>
>  > >
>  > >  > First commit message:
>  > >  >   http://marc.theaimsgroup.com/?l=openbsd-cvs&m=111692513727274&w=2
>  > >  >
>  > >  > The MFC with all changes in one commit message:
>  > >  >   http://marc.theaimsgroup.com/?l=openbsd-cvs&m=111859519015510&w=2
>  > >
>  > > The bug they fixes was missing interrupts by both calling APM's idle
>  > > routine, which may hlt, and hlt'ing in the idle loop itself.  Since I
>  > > have no idea what acpi is doing, I got excited about this.
>  > >
>  > > Alas, it seems like this isn't it.  I pointed cpu_idle_hook back to
>  > > cpu_idle_default and away from acpi_cpu_idle, but that made no
>  > > difference.
>  >
>  > You may be seeing problems because it might simply take a while for the
>  > CPU to wake up from HLT when an interrupt comes in.  The 4BSD scheduler
>  > tries to do IPIs to wakeup any sleeping CPUs when it schedules a new
>  > thread, but that would add higher latency for ithreads than just
>  > preempting directly to the ithread.  Oh, you have to turn that on, it's
>  > off by default
>  > (kern.sched.ipiwakeup.enabled=1).
>
> Hmm..  It seems to be on by default.  Unfortunately, it does not seem
> to help.

I'm not sure.

> Would you expect ULE to do better?  I've noticed that if I screw up
> the time state of the machine by switching between ACPI-fast and TSC
> timecounters, performance for TCP ping-pongs goes all over the map...

No ideas on this one. :(

-- 
John Baldwin <jhb at FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org