Timer problems on FreeBSD 11R on KVM

[Dmitry Marakasov]

* Allan Jude (allanjude at freebsd.org) wrote:

> > I'm running a bunch of FreeBSD VMs on VPS provider which uses KVM.
> > Recently one of them began bahaving strangely. Symptoms:
> > 
> > - Much ~(30%) CPU spent in intr
> > - top shows CPU used by "intr{swi4: clock (0)}"
> > - kernel: calcru: runtime went backwards from 13958 usec to 10792 usec for pid 919 (sshd)
> >   messages in logs
> > - There's no much load, no high network traffic, systat shows no excess
> >   interrupts.
> > - This intr load is sporadic in form of ~30 minute busts each several
> >   hours
> > - The box had ntpd running, but disabling it doesn't help
> > - Playing with timer-related sysctls does not help either
> > 
> > I've tried i8254 and TSC-low timecounters:
> > 
> > kern.timecounter.hardware: ACPI-fast
> > kern.timecounter.choice: TSC-low(800) i8254(0) ACPI-fast(900) dummy(-1000000)
> > 
> > and kern.eventtimer.periodic=1
> > 
> > to no effect.
> > 
> > 
> > I suspect that this began after some changes on the hoster's side.
> > 
> > - This VM was working for more than a year without problems before
> > - A graph which monitors a reply time of website running on that VM
> >   showed min. response time jump from 50 to 75 msec on Monday, after
> >   which problems began.
> > 
> > So while I'm getting the hoster to tell me what they have changed, any
> > ideas?
> > 
> 
> Look at:
> kern.eventtimer.choice
> 
> and try different kern.eventtimer.timer values

Both eventtimer and timecounter i8254 haven't helped. For some days it
was OK, then another intrrupt load. Though at least it doesn't kill the
box, as with eventtimer LAPIC and timecounter i8254 the machine is
likely to die as in it does respond to ping, but not HTTP and SSH.
Is RTC worth trying? And is there a way to profile what is the kernel
doing in intr in the moments of load?

-- 
Dmitry Marakasov   .   55B5 0596 FF1E 8D84 5F56  9510 D35A 80DD F9D2 F77D
amdmi3 at amdmi3.ru  ..:  jabber: amdmi3 at jabber.ru      http://amdmi3.ru