instability of timekeeping

[Andriy Gapon]

On 29/10/2015 12:56, Andriy Gapon wrote:
> Okay, more data:
> https://people.freebsd.org/~avg/timekeeping-ktr.2.patch
> https://people.freebsd.org/~avg/timekeeping.2.ktrdump.txt
> 
> I think that the snippet (amended with some notes of mine) makes it painfully
> obvious that the timer interrupt got very delayed when all CPUs entered the idle
> state.
> I do not see anything that could suggest a FreeBSD bug.
> 
> There is another sad discovery.  Turns out that my CPU model provides two ways
> of doing C1E magic.  The sane one: the north bridge logic in the CPU performs a
> read of a configured LVL3 register so that C3 is entered.  The insane one: the
> CPU NB performs a write of a configured value to a configured SMI register, so
> that the SMI is generated and an SMM handler does the job (probably reading from
> LVL2 or LVL3).  Looking at MSR C001_0055 I see that my BIOS has chosen the
> insane approach[*], quite unfortunately.  Bugs in the SMM code are not unheard
> of, to put it mildly, so that could be an explanation for my problem.
> 
> So, I guess I'll just disable C1E and end this investigation.

Just in case anyone is still interested in this, I have found that a newer BIOS
version is available for my motherboard and it includes a newer version of
AGESA.  So, I upgraded the BIOS and the new version does C1E in the sane way:

$ cpucontrol -m 0xc0010055 /dev/cpuctl0
MSR 0xc0010055: 0x00000000 0x14c14015

C1eOnCmpHalt: C1E on chip multi-processing halt.
 - read from 0x4015

Also, after a few hours of testing I do not see any problems with the
timekeeping.  So, either the problem was indeed in the SMM code in the older
BIOS or the older BIOS incorrectly configured the sleep break events.

Unfortunately, the newer BIOS has a regression in a different area: it fails to
access HDD LBAs greater than INT32_MAX (offsets beyond 1TB on my HDDs with
512-byte native sector sizes).  But that's a different story.

> [*]
> $ cpucontrol -m 0xc0010055 /dev/cpuctl0
> 
> 
> MSR 0xc0010055: 0x00000000 0x083400b0
> 
> SmiOnCmpHalt: SMI on chip multi-processing halt.
>  - write 0x34 to port 0xb0
> 


-- 
Andriy Gapon