VirtualBox 4.2.4 on FreeBSD 9.1-PRERELEASE problem: VMs behave very different when pinned to different cores

[Andriy Gapon]

on 28/11/2012 17:34 Alex Chistyakov said the following:
> On Wed, Nov 28, 2012 at 7:24 PM, Andriy Gapon <avg at freebsd.org> wrote:
>> on 26/11/2012 09:10 Alex Chistyakov said the following:
>>> CPU: Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz (3200.18-MHz K8-class CPU)
>>>   Origin = "GenuineIntel"  Id = 0x206d7  Family = 0x6  Model = 0x2d
>>> Stepping = 7
>>> Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
>>> Features2=0x1fbee3bf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX>
>>>   AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
>>>   AMD Features2=0x1<LAHF>
>>>   TSC: P-state invariant, performance statistics
>>>
>>
>> Is this a multi-socket system?
> 
> No, this is a single-socket desktop grade baseboard, Intel DX79TO.
> 

I'd say that there must be some kind of hardware or firmware (BIOS) problem to
create such a discrepancy in TSC readings.

>> It would be very strange that a modern CPU like this would have such a skew
>> between TSC on different cores.
>>
>> On my Core i5-3570 I see that the _observed_ skew is no more than 100 ticks (after
>> many days of uptime).  It could be zero, in fact, given the inaccuracy of
>> inter-core measurements.

You could try the following (totally untested) patch as an experiment.  Just to
see if indeed there is some connection between the TSC issue and the main issue.

--- a/sys/x86/x86/tsc.c
+++ b/sys/x86/x86/tsc.c
@@ -398,6 +398,13 @@ comp_smp_tsc(void *arg)
 		}
 }

+static void
+reset_tsc(void *arg)
+{
+
+	wrmsr(MSR_TSC, (uintptr_t)arg);
+}
+
 static int
 test_smp_tsc(void)
 {
@@ -406,6 +413,9 @@ test_smp_tsc(void)

 	if (!smp_tsc && !tsc_is_invariant)
 		return (-100);
+
+	smp_rendezvous(NULL, reset_tsc, NULL, (void*)0);
+
 	size = (mp_maxid + 1) * 3;
 	data = malloc(sizeof(*data) * size * N, M_TEMP, M_WAITOK);
 	for (i = 0, tsc = data; i < N; i++, tsc += size)

-- 
Andriy Gapon