time issues and ZFS

[Ian Lepore]

On Mon, 2013-01-21 at 13:33 +0200, Daniel Braniss wrote:
> After many trials (and errors), here are some facts:
> 
> host: DELL PowerEdge R710, 16GB, 
>  mfi0: <Dell PERC H700 Integrated>
>  mfid0: 14305280MB (29297213440 sectors) RAID volume 'r5' is optimal
>  mfi1: <Dell PERC 6> 
>  mfid1: 12393472MB (25381830656 sectors) RAID volume 'Virtual Disk 0' is 
> optimal
> 
> we have NO problems with FreeBSD-8.3-STABLE, but with 9.1-STABLE, the real-time
> clock slows down when doing some zfs stuff like send|receive, typing 'date' 
> when less that 1000s went by seems to crorrect the problem,
> ntpd kicks in and on track again.
> 
> I have a cron job just logging date every 5 minutes, and the loghost sees:
> 
> |-- local time on loghost             | time on problematic host
> Jan 20 19:56:19 store-02.cs.huji.ac.il Jan 20 19:56:19 danny: Sun Jan 20 
> 19:56:19 IST 2013	-- ok
> Jan 20 20:15:00 store-02.cs.huji.ac.il Jan 20 20:15:00 danny: Sun Jan 20 
> 20:15:00 IST 2013	-- ok
> Jan 20 21:30:00 store-02.cs.huji.ac.il Jan 20 20:21:06 danny: Sun Jan 20 
> 20:21:06 IST 2013	-- off by 1:09
> Jan 20 21:33:53 store-02.cs.huji.ac.il Jan 20 20:25:00 danny: Sun Jan 20 
> 20:25:00 IST 2013	-- off by 1:08
> Jan 20 21:38:54 store-02.cs.huji.ac.il Jan 20 20:30:00 danny: Sun Jan 20 
> 20:30:00 IST 2013	-- off by 1:09
> ...
> Jan 20 22:03:54 store-02.cs.huji.ac.il Jan 20 20:55:00 danny: Sun Jan 20 
> 20:55:00 IST 2013	-- diff is now constant
> ..
> Jan 20 22:04:13 store-02.cs.huji.ac.il Jan 20 20:55:19 ntpd[1848]: time 
> correction of 4134 seconds exceeds sanity limit (1000); set clock manually to 
> the correct UTC time.
> ...
> Jan 20 22:58:53 store-02.cs.huji.ac.il Jan 20 21:50:00 danny: Sun Jan 20 
> 21:50:00 IST 2013
> 
> 
> strangely, when running 8.3, ACPI-fast is chosen:
> 	kern.timecounter.choice: TSC(-100) HPET(900) ACPI-fast(1000) i8254(0) 
> dummy(-1000000)
> but with 9.1 TSC-low gets chosen:
> 	kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) HPET(950) i8254(0) 
> dummy(-1000000)
> 
> so I did sysctl kern.timecounter.hardware=ACPI-fast, but the same happens - 
> unless it can't be changed after boot.
> 
> I realy need help here!
> 
> thanks,
> 	danny

What's the output of sysctl kern.eventtimer?  Does the bad behavior
change if you set kern.eventimer.periodic=1?

-- Ian