8.1-PRERELEASE: CPU packages not detected correctly

[Jung-uk Kim]

On Thursday 19 August 2010 03:30 pm, pluknet wrote:
> On 19 August 2010 21:26, Jung-uk Kim <jkim at freebsd.org> wrote:
> > On Thursday 19 August 2010 12:56 pm, pluknet wrote:
> >> On 19 August 2010 20:39, Andriy Gapon <avg at icyb.net.ua> wrote:
> >> > on 10/08/2010 19:55 pluknet said the following:
> >> >> On 16 July 2010 19:47, Jung-uk Kim <jkim at freebsd.org> wrote:
> >> >>> The patch should apply fine on both
> >> >>> sys/amd64/amd64/mp_machdep.c and sys/i386/i386/mp_machdep.c.
> >> >>>
> >> >>> http://people.freebsd.org/~jkim/mp_machdep2.diff
> >> >>
> >> >> Hi.
> >> >>
> >> >> Just checked on Xen HVM with 3 cores.
> >> >> 1) 8.1 unmodified:
> >> >> FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs
> >> >> FreeBSD/SMP: 1 package(s) x 3 core(s)
> >> >>
> >> >> 2) 8.1 + patch
> >> >> FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs
> >> >> FreeBSD/SMP: 0 package(s) x 1 core(s) x 32 HTT threads
> >> >> WARNING: Non-uniform processors.
> >> >> WARNING: Using suboptimal topology.
> >> >
> >> > Can you debug, e.g. with printfs, what exactly goes wrong?
> >> > I wonder if in this case code follows some unusual/unexpected
> >> > path.
> >>
> >> Sorry, I'm a bit busy right now.
> >> I hope to debug this somewhere in the next week.
> >>
> >> > BTW, could you please also provide CPU name/model/features as
> >> > detected by the kernel?
> >>
> >> Sure.
> >> CPU: Intel(R) Xeon(R) CPU           E5520  @ 2.27GHz
> >> (2763.12-MHz 686-class CPU) Origin = "GenuineIntel"  Id =
> >> 0x106a5  Family = 6 Model = 1a  Stepping = 5
> >> Features=0x1781fbbf<FPU,VME,DE,PSE,TSC,MSR,MCE,CX8,APIC,SEP,MTRR
> >>,PG E,MCA,CMOV,PAT,MMX,FXSR,SSE,SSE2,HTT>
> >> Features2=0x80982201<SSE3,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,<b31>>
> >> TSC: P-state invariant
> >> real memory  = 4194304000 (4000 MB)
> >> avail memory = 3932786688 (3750 MB)
> >> ACPI APIC Table: <Xen HVM>
> >> FreeBSD/SMP: Multiprocessor System Detected: 3 CPUs
> >> FreeBSD/SMP: 0 package(s) x 1 core(s) x 32 HTT threads
> >>  cpu0 (BSP): APIC ID:  0
> >>  cpu1 (AP/HT): APIC ID:  2
> >>  cpu2 (AP/HT): APIC ID:  4
> >>
> >> Just a thought.
> >>  # HTT might somehow correlate with current maxcpus limit (32).
> >
> > One thing I am not sure is whether those CPUID instructions are
> > executed on *real* CPUs or translated in HVM.
>
> I may add only that <b31> of Features2 presents only in Xen
> HVM environment, and its role is afaik to indicate a Xen guest
> mode. There is no any mention of this bit in the latest Intel doc
> (ie it's reserved/unused).

Ah, that means the HVM is actually translating the instruction, not 
running directly on the CPU.  That means, we cannot use any CPUID 
instructions for topology detection in HVM.  And I bet all MSRs will 
be translated as well.  It is not just HVM, but also any emulations 
and virtualizations in general.  Actually, CPU topology detection 
does nothing in these environments because hypervisor or whatever 
will do memory translations and stuff unless some "hints" are given 
by guest or "ballooning" is done for virtual devices and resources.  
Usually, this kind of problem is handled by VM-specific device 
drivers, e.g., VirtualBox Guest Additions, VMware Tools, etc.  In 
theory, Xen domU should do much better job than these tools because 
it is specifically modified to handle these problems without losing 
performance.

> Also, at least NetBSD has a special handling of this bit.
> See commit log for CPUID2_RAZ in sys/arch/x86/include/specialreg.h,
> 1.37 FWIW RAZ states for "reserved and zero" or so.

Hmm...  Interesting.  But we should never rely on an undocumented bit 
to identify HVM or whatever.

Thanks for the info,

Jung-uk Kim