RFC: setting performance_cx_lowest=C2 in -HEAD to avoid lock contention on many-CPU boxes

[Davide Italiano]

On Sat, Apr 25, 2015 at 9:31 AM, Adrian Chadd <adrian at freebsd.org> wrote:
> Hi!
>
> I've been doing some NUMA testing on large boxes and I've found that
> there's lock contention in the ACPI path. It's due to my change a
> while ago to start using sleep states above ACPI C1 by default. The
> ACPI C3 state involves a bunch of register fiddling in the ACPI sleep
> path that grabs a serialiser lock, and on an 80 thread box this is
> costly.
>
> I'd like to drop performance_cx_lowest to C2 in -HEAD. ACPI C2 state
> doesn't require the same register fiddling (to disable bus mastering,
> if I'm reading it right) and so it doesn't enter that particular
> serialised path. I've verified on Westmere-EX, Sandybridge, Ivybridge
> and Haswell boxes that ACPI C2 does let one drop down into a deeper
> CPU sleep state (C6 on each of these). I think is still a good default
> for both servers and desktops.
>
> If no-one has a problem with this then I'll do it after the weekend.
>

This sounds to me just a way to hide a problem.
Very few people nowaday run on NUMA and they can tune the machine as
they like when they do testing.
If there's a lock contention problem, it needs to be fixed and not
hidden under another default.
Also, as already noted this is a problem on 80-core machines but
probably not on a 2-core Atom. I think you need to understand factors
better and come up with a more sensible relation. In other words, your
bet needs to be proven before changing a default useful for frew that
can impact many.

-- 
Davide

"There are no solved problems; there are only problems that are more
or less solved" -- Henri Poincare