svn commit: r237434 - in head/lib/libc: amd64/sys gen i386/sys include sys

[Marius Strobl]

On Sat, Jun 23, 2012 at 05:05:56PM +0300, Konstantin Belousov wrote:
> On Sat, Jun 23, 2012 at 03:17:57PM +0200, Marius Strobl wrote:
> > On Fri, Jun 22, 2012 at 10:48:17AM +0300, Konstantin Belousov wrote:
> > > On Fri, Jun 22, 2012 at 09:34:56AM +0200, Marius Strobl wrote:
> > > > On Fri, Jun 22, 2012 at 07:13:31AM +0000, Konstantin Belousov wrote:
> > > > > Author: kib
> > > > > Date: Fri Jun 22 07:13:30 2012
> > > > > New Revision: 237434
> > > > > URL: http://svn.freebsd.org/changeset/base/237434
> > > > > 
> > > > > Log:
> > > > >   Use struct vdso_timehands data to implement fast gettimeofday(2) and
> > > > >   clock_gettime(2) functions if supported. The speedup seen in
> > > > >   microbenchmarks is in range 4x-7x depending on the hardware.
> > > > >   
> > > > >   Only amd64 and i386 architectures are supported. Libc uses rdtsc and
> > > > >   kernel data to calculate current time, if enabled by kernel.
> > > > 
> > > > I don't know much about x86 CPUs but is my understanding correct
> > > > that TSCs are not synchronized in any way across CPUs, i.e.
> > > > reading it on different CPUs may result in time going backwards
> > > > etc., which is okay for this application though?
> > > 
> > > Generally speaking, tsc state among different CPU after boot is not
> > > synchronized, you are right.
> > > 
> > > Kernel has somewhat doubtful test which verifies whether the after-boot
> > > state of tsc looks good. If the test fails, TSC is not enabled by
> > > default as timecounter, and then usermode follows kernel policy and
> > > falls back to slow syscall. So we err on the safe side.
> > > I tested this on Core i7 2xxx, where the test (usually) passes.
> > 
> > Okay, so for x86 the TSCs are not used as timecounters by either
> > the kernel or userland in the SMP case if they don't appear to
> > be synchronized, correct?
> Correct as for now.  But this is bug and not a feature. The tscs shall
> be synchronized, or skew tables calculated instead of refusing to use it.
> 
> > 
> > > 
> > > While you are there. do you have comments about sparc64 TICK counter ?
> > > On SMP, the counter of BSP is used by IPI. Is it unavoidable ?
> > 
> > The TICK counters are per-core and not synchronized by the hardware.
> > We synchronize APs with the BSP on bring-up but they drift over time
> > and the initial synchronization might not be perfect in the first
> > place. At least in the past, drifting TICK counters caused all sorts
> > of issues and strange behavior in FreeBSD when used as timecounter
> > in the SMP case. If my understanding of the above is right, as is
> > this still rules them out as timecounters for userland.
> > Linux has some complex code (based on equivalent code origining in
> > their ia64 port) for constantly synchronizing the TICK counters.
> > In order to avoid that complexity and overhead, what I do in
> > FreeBSD in the SMP case is to (ab)use counters (either intended
> > for that purpose or bus cycle counters probably intended for
> > debugging the hardware during development) available in the
> > various host-to-foo bridges so it doesn't matter which CPU they
> > are read by. This works just fine except for pre-PCI-Express
> > based USIIIi machines, where the bus cycle counters are broken.
> > That's where the TICK counter is always read from the BSP
> > using an IPI in the SMP case. The latter is done as sched_bind(9)
> > isn't possible with td_critnest > 1 according to information
> > from jhb@ and mav at .
> > So apart from introducing code to constantly synchronize the
> > TICK counters, using the timecounters on the host busses also
> > seems to be the only viable solution for userland. The latter
> > should be doable but is long-winded as besides duplicating
> > portions of the corresponding device drivers in userland, it
> > probably also means to get some additional infrastructure
> > like being able to memory map registers for devices on the
> > nexus(4) level in place ...
> 
> Understand. I do plan eventually to map HPET counters page into usermode
> on x86.
> 
> Also, as I noted above, some code to synchronize per-package counters
> would be useful for x86, so it might be developed with multi-arch
> usage in mind.

That would be great.

Marius