powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed]

[Konstantin Belousov]

On Fri, Apr 05, 2019 at 11:52:27PM +1100, Bruce Evans wrote:
> On Fri, 5 Apr 2019, Konstantin Belousov wrote:
> 
> > On Thu, Apr 04, 2019 at 02:47:34AM +1100, Bruce Evans wrote:
> >> I noticed (or better realized) a general problem with multiple
> >> timehands.  ntpd can slew the clock at up to 500 ppm, and at least an
> >> old version of it uses a rate of 50 ppm to fix up fairly small drifts
> >> in the milliseconds range.  500 ppm is enormous in CPU cycles -- it is
> >> 500 thousand nsec or 2 million cycles at 4GHz.  Winding up the timecounter
> >> every 1 msec reduces this to only 2000 cycles.
> >> ...
> >> The main point of having multiple timehands (after introducing the per-
> >> timehands generation count) is to avoid blocking thread N during the
> >> update, but this doesn't actually work, even for only 2 timehands and
> >> a global generation count.
> >
> > You are describing the generic race between reader and writer. The same
> > race would exist even with one timehand (and/or one global generation
> > counter), where ntp adjustment might come earlier or later of some
> > consumer accessing the timehands. If timehand instance was read before
> > tc_windup() run but code consumed the result after the windup, it might
> > appear as if time went backward, and this cannot be fixed without either
> > re-reading the time after time-depended calculations were done and
> > restarting, or some globabl lock ensuring serialization.
> 
> With 1 timehand, its generation count would be global.  I think its ordering
> is strong enough to ensure serialization.
Yes, single timehands result in global generation.  But it would not solve
the same race appearing in slightly different manner, as I described above.
If reader finished while generation number in th was not yet reset, but
caller uses the result after tc_windup(), the effect is same as if we
have two th's and reader used the outdated one.

> 
> I think the fix in the kernel to use a global generation count (with > 1
> timehands) is simply s/th->th_generation/tc_generation/g.  Oops, that
> makes multiple timehands useless and gives some blocking.  The critical
> case is when a new timehands is under construction.  The old timehands
> becomes unsafe to use when the writer (tc_windup()) updates the offset.
> tc_windup() currently sets th_generation to 0 to indicate that the new
> timehands is unsafe to use.  Doing the same with a global tc_generation
> would give serialization at the cost of busy-waiting for tc_generation
> to become nonzero.  It would indicate that all timehands are unsafe
> to use.
> 
> In the library, does it just work to put the global generation count in
> the shared page?
libc always reload tk_current in the loop, so it works with any number
of vdso timehands greater or equal to one.