svn commit: r222853 - in head: . sys/amd64/amd64 sys/amd64/include sys/i386/i386 sys/i386/include sys/pc98/pc98

[Ryan Stone]

On Wed, Jun 8, 2011 at 4:12 AM, Andriy Gapon <avg at freebsd.org> wrote:
> Author: avg
> Date: Wed Jun  8 08:12:15 2011
> New Revision: 222853
> URL: http://svn.freebsd.org/changeset/base/222853
>
> Log:
>  remove code for dynamic offlining/onlining of CPUs on x86
>
>  The code has definitely been broken for SCHED_ULE, which is a default
>  scheduler.  It may have been broken for SCHED_4BSD in more subtle ways,
>  e.g. with manually configured CPU affinities and for interrupt devilery
>  purposes.

I can confirm that this was very broken for SCHED_4BSD.  The softclock
threads are kicked off from hardclock_cpu().  When you offline a CPU,
hardclock_cpu() never runs for that CPU so its softclock thread is
never run.  If you happen to have any callouts scheduled on that
softclock thread they will never run unless the corresponding CPU is
brought online again.

In a particular case that I've been looking at, the
hyperthreading_allowed tunable was set to 0 on a system running a
derivative of 8.2-RELEASE.  Unfortunately it seems that there is a
window in SCHED_4BSD during which the hyperthreads run even if you
disable them via tunable.  It seems that bufdaemon ran on one of the
hyperthreads and then went to sleep.  It's thread's td_slpcallout was
scheduled on the hyperthread's softclock thread, and then 4BSD finally
got around to disabling the hyperthreads.  Hilarity ensues as
bufdaemon never wakes up.

In short, there's currently no safe way to disable hyperthreading on
stable/8.  I know that you and Attilio originally decided not to MFC
this as these sysctls were considered a part of the ABI, but seeing as
the functionality that this ABI is intended to present is completely
broken, I'm wondering if an MFC is the lesser evil.