svn commit: r297039 - head/sys/x86/x86

[Konstantin Belousov]

On Wed, Mar 23, 2016 at 02:21:59PM -0700, John Baldwin wrote:
> As you noted, the issue is if a timecounter needs locks (e.g. i8254) though
> outside of that I think the patch is great. :-/  Of course, if the TSC
> isn't advertised as invariant, DELAY() is talking to the timecounter
> directly as well.
> 
> However, I think we probably can use the TSC.  The only specific note I got
> from Ryan (cc'd) was about the TSC being unstable as a timecounter under KVM.
> That doesn't mean that the TSC is non-mononotic on a single vCPU.  In fact,
> thinking about this more I have a different theory to explain how the TSC
> can be out of whack on different vCPUs even if the hardware TSC is in sync
> in the physical CPUs underneath.
In fact, if we can use TSC with the only requirement of being monotonic,
I do not see why do we need TSC at all. We can return to pre-r278325
loop, but calibrate the number of loop iterations for known delay in
1us, once on boot.  Do you agree with this ?

> 
> One of the things present in the VCMS on Intel CPUs using VT-x is a TSC
> adjustment.  The hypervisor can alter this TSC adjustment during a VM-exit to
> alter the offset between the TSC in the guest and the "real" TSC value in the
> physical CPU itself.  One way a hypervisor might use this is to try to
> "pause" the TSC during a VM-exit by taking TSC timestamps at the start and
> end of a VM-exit and adding that delta to the TSC offset just before each
> VM-entry.  However, if you have two vCPUs, one of which is running in the
> guest and one of which is handling a VM-exit in the hypervisor, the TSC on
> the first vCPU will run while the effective TSC of the second vCPU is paused.
> When the second vCPU resumes after a VM-entry, it's TSC will now "unpause",
> but it will lag the first vCPU by however long it took to handle its VM-exit.
> 
> It wouldn't surprise me if KVM was doing this.  bhyve does not do this to my
> knowledge (so the TSC is actually still usable as a timecounter under bhyve
> for some value of "usable").  However, even with this TSC pausing/unpausing,
> the TSC would still increase monotonically on a single vCPU.  For the purposes
> of DELAY() (and other spin loops on a pinned thread such as in
> lapic_ipi_wait()), that is all you need.
BTW, Intel exported this mechanism to the non-VT environment as well, on
recent chips.  So I would be not too surprised if SMM handlers start
'compensate' for some long delays in near future.