kern/145385: [cpu] Logical processor cannot be disabled for some SMT-enabled Intel procs

[Garrett Cooper]

The following reply was made to PR kern/145385; it has been noted by GNATS.

From: Garrett Cooper <gcooper at freebsd.org>
To: Andriy Gapon <avg at freebsd.org>
Cc: bug-followup at freebsd.org
Subject: Re: kern/145385: [cpu] Logical processor cannot be disabled for some
 SMT-enabled Intel procs
Date: Thu, 18 Nov 2010 16:09:46 -0800

 On Wed, Nov 17, 2010 at 4:34 PM, Garrett Cooper <gcooper at freebsd.org> wrote=
 :
 > On Thu, Oct 7, 2010 at 10:03 AM, Andriy Gapon <avg at freebsd.org> wrote:
 >>
 >> Here's a patch to remove halted logical processors from root cpu set (cp=
 uset
 >> number zero) and consequently all child sets:
 >> http://people.freebsd.org/~avg/cpu-halt.diff
 >>
 >> Please note that unhalting CPUs will add them to cpuset zero, but will n=
 ot
 >> (re-)add them cpusets of the running processes. =A0So additional action =
 will be
 >> required from system administrator if e would like existing processes to=
  make use
 >> of unhalted CPUs.
 >>
 >> Also, interrupts that are bound to halted CPUs are not rebound on halt, =
 and so
 >> will be delivered to halted CPUs. =A0This should not be a problem for ty=
 pical
 >> environments, but would be nice to fix anyway.
 >
 > =A0 =A0Sorry for taking so long to get back on this item. The patch looks
 > ok as far as setting the CPUSET is concerned (setting
 > machdep.hlt_logical_cpus=3D1 works on an r710 with 8 logical procs as
 > SCHED_ULE schedules all of the tasks on the non-logical SMT cores, not
 > the logical ones, and the patch properly detects that there aren't any
 > logical processors on a Core2 Quad I have at home), but it's missing a
 > key case where I go from...
 >
 > machdep.hlt_logical_cpus: 1 -> 0
 >
 > =A0 =A0... it should reset CPUSETZERO.
 > =A0 =A0The overall CPU topology should probably be cached and restored
 > when reset, or at least CPUSETZERO should be reset (probably the
 > simpler and cleaner route to go).
 > =A0 =A0After that all that's required is work for i386 and I'd vote for a
 > commit of the patch.
 > Thanks for the hard work Andriy :)!
 
 Just to illustrate, here's how I reproduced the successful behavior
 (and the problem):
 
 1. Execute the following script:
 
 #!/bin/sh
 i=3D0
 ncpus=3D`sysctl -n kern.smp.cpus`
 while [ $i -lt $ncpus ] ; do
     while : ; do : ; done &
     : $(( i +=3D 1 ))
 done
 
 2. Execute sysctl machdep.hlt_logical_cpus=3D1 .
 3. Look at cpuset info; example:
 
 Before:
 
 # for i in `ps aux | grep gcooper | grep -v grep | awk '{ print $2 }'`
 ; do cpuset -g -p $i ; done
 pid 2179 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15
 pid 2180 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15
 
 After:
 
 # for i in `ps aux | grep gcooper | grep -v grep | awk '{ print $2 }'`
 ; do cpuset -g -p $i ; done
 pid 2179 mask: 0, 2, 4, 6, 8, 10, 12, 14
 pid 2180 mask: 0, 2, 4, 6, 8, 10, 12, 14
 # sysctl machdep.hlt_logical_cpus=3D0
 machdep.hlt_logical_cpus: 1 -> 0
 # for i in `ps aux | grep gcooper | grep -v grep | awk '{ print $2 }'`
 ; do cpuset -g -p $i ; done
 pid 2179 mask: 0, 2, 4, 6, 8, 10, 12, 14
 pid 2180 mask: 0, 2, 4, 6, 8, 10, 12, 14
 
 I'm taking the patch and running with it now to try and get it working 100%=
 .