get_cyclecount(9) deprecation

[Jung-uk Kim]

On Friday 18 March 2011 04:11 pm, Kostik Belousov wrote:
> On Fri, Mar 18, 2011 at 02:09:58PM -0400, Jung-uk Kim wrote:
> > On Friday 18 March 2011 01:05 pm, Bruce Evans wrote:
> > > On Sat, 19 Mar 2011, Bruce Evans wrote:
> > > > On Fri, 18 Mar 2011, Kostik Belousov wrote:
> > > >> We definitely do not support configurations with different
> > > >> models of CPUs in SMP, this is what Simmetric is about.
> > > >> Different as in frequency or stepping.
> > > >
> > > > ...
> > > > Now there is even more asymmetry
> > > > in core frequencies, with the hardware transiently slowing
> > > > down or stopping cores independently, at least for cores in
> > > > different packages.
> > >
> > > Also, with virtualization, the virtualizer cannot reasonably
> > > even provide an invariant TSC that runs at the same rate on all
> > > cores. It should provide an invariant TSC that claims to run at
> > > the same rate on all cores, but then the cores cannot run at
> > > the same rate except on average, since some of the cores will
> > > have to run the virtualizer some of the time, and it is
> > > unreasonable to distribute the overhead for this evenly except
> > > on average.
> >
> > Ah, virtualization...  It is really painful to make it reasonably
> > correct.  For example, VMware claims that all timers (including
> > TSC and excluding RTC) runs at "apparent time", which is concept
> > of constant ticks in virtualized guest.  It also means TSCs are
> > always "virtually" constant and synchronized across all virtual
> > cores in guest environment.  If it loses periodic timer ticks,
> > lost ticks are "compensated" later.  This also means timecounter
> > does not exactly scale well based on realtime and its frequency
> > fluctuates so much, if my understanding is correct:
> >
> > http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pd
> >f
> >
> > I am not sure how others handle this but VirtualBox gives me
> > really wacky TSC frequency sometimes.  When it happens, it
> > becomes unusably slow.  So, I know something is really wrong
> > there, too.  However, Xen seems to do much smarter than that
> > because it has its own concept of virtual TSC, thanks to its
> > para-virtualization architecture.
>
> Most likely, all 'full-hardware' hypervisors have to disable rdtsc
> for guests, intercepting the instruction by trap and emulating it.
> This means that most basic assumptions about rdtsc are not held
> in virtualized environment, like relative cost or accuracy.

That's exactly what's happening with "hardware-assisted" 
virtualization.  If it is pure emulation, it may run on arbitrary 
host CPU (not-translated x86 on x86) or completely artificial 
(translated) depending on emulators.  Therefore, it has zero 
advantage over other timers.  Sometimes, it may be worse.

Good news is I added a tunable "machdep.disable_tsc" exactly for that 
reason. :-)

> Anyway, I was unable to make any reasonable use of virtualization
> except kernel debugging, which is more then satisfactory performed
> by QEMU. Ah, QEMU is not hypervisor.

Yeah, I've used QEMU for years for the same reason but its timing is 
awfully wrong and SMP is not exactly SMP there. :-(

Jung-uk Kim