Synchronizing TSC

[Jim Harris]

On Tue, Apr 16, 2013 at 3:04 PM, Alexander Motin <mav at freebsd.org> wrote:

> Hi.
>
> Recently I've got 6-core/12-thread system on Sandy Bridge-E Core i7-3930K
> CPU and was unpleasantly surprised to see that TSCs are not synchronized
> there. While all 11 APs were synchronized, BSP was far behind them. Since
> it is single-socket system, I don't know any good reason for such behavior
> except some BIOS bug. But I've recalled that somewhere was some discussions
> about possible TSC synchronization. I've implemented patch below that
> allows to adjust TSC values of BSPs to AP's one on boot using CPU MSRs,
> hoping that they should not diverge after that:
> http://people.freebsd.org/~**mav/tsc_adj2.patch<http://people.freebsd.org/~mav/tsc_adj2.patch>
>
> I don't know very much about all different TSC hardware to predict when it
> is safe to enable the functionality, but at least on my system being
> enabled via loader tunable it seems working well.
>
> Comments?
>
>
You may be remembering this thread on r238755 last year:

http://lists.freebsd.org/pipermail/svn-src-head/2012-July/038992.html

This was a bug fix in the TSC synchronization test code though, not
anything for trying to adjust out-of-sync TSCs.

The Intel SDM (volume 3, section 17.13 of March 2013 revision) says earlier
models can only write to lower 32 bits of IA32_TIME_STAMP_COUNTER, but
these models also should not have invariant TSC so they would never even
get to your new routine.  So your patch seems OK for Intel CPUs, at least
as a tunable that is disabled by default.

My only concern would be why TSC on the BSP started out-of-sync on your
system.  Theoretically, BIOS could adjust TSCs in SMM to try to hide SMI
code execution from the OS, which could then make them out-of-sync again.
Not sure if that's what's happening here, but might be worth a test putting
the TSC test code on a periodic timer to see if they ever get out of sync
again.

-Jim