Is it a good idea to use a usb-serial adapter for PPS input? Yes, it is.

[Ian Lepore]

Since we added support for accepting a PPS signal on a USB-serial
adapter a couple years ago, I've seen people express reluctance to use
it several times.  Usually they cite concerns about latency and
jitter.  I decided it was time to do some rigorous testing, and post
the results.  

Complete details on the test setup appear below.  The tl;dr summary is:

A single PPS source is fanned out to 4 different types of inputs on a
Wandboard system (armv7, imx6 SoC).  Ntpd is configured to steer only
to PPS(0), and the offset and jitter numbers for the other sources
allow comparing the various PPS processing paths.  After running about
12 hours, the results are: 

     remote      refid  st t when poll reach   delay   offset  jitter
=====================================================================
oPPS(0)          .gpt.   0 l    8   16  377    0.000    0.000   0.002
 PPS(1)          .gpio.  0 l    7   16  377    0.000   -0.002   0.002
 PPS(2)          .usb1.  0 l    6   16  377    0.000   -0.201   0.034
 PPS(3)          .usb2.  0 l    5   16  377    0.000   -0.215   0.026
*tflex.my.lan    .GPS.   1 u   32   64  377    0.568    0.038   0.061

PPS(0) is fed to a hardware timer block within the imx6 SoC.  The PPS
pulse triggers hardware capture of the kernel clock, eliminating
latency and jitter due to interrupt processing.  

PPS(1) is fed to a generic gpio input pin on the SoC, handled by the
standard gpiopps driver processing a pin-change interrupt.  

PPS(2) is an FTDI 232R, a USB 1.1 serial adapter, connected to a port
on a USB 2.0 hub that's connected to a USB 2.0 host port on the
Wandboard.  

PPS(3) is an FTDI 4232H, a USB 2.0 serial adapter, connected to a port
on the same USB hub as PPS(2).  

Unfortunately, the uart driver for the imx6 SoC doesn't support the CTS
or CD signals, so I couldn't measure native uart performance
directly.  I would expect the performance to be comparable to the gpio
pin input.  

As shown above, there is a 2 microsecond latency and virtually no
jitter on the GPIO input.  The USB 1.1 and USB 2.0 adapters performed
essentially identically to each other, with about 200 microseconds of
latency and negligible jitter.  There was no difference in performance
between using the CTS versus the CD pins on the adapters for input.  

To see if lots of USB bus activity increased latency or noise, I
connected a USB SATA dock containing an SSD drive to the same USB hub
as the serial adapters, and ran a continuous dd(1) from the drive to
/dev/null.  Surprisingly, there was absolutely no difference in the
results during that run.  

Most people are not worried about their kernel clock being 200
microseconds off from UTC, even if they're using the PPS signal from a
GPS receiver.  So I think most people should feel completely at ease
using a USB serial adapter as the input device for a PPS signal.  


Test setup details...


PPS measurements are made using the kernel clock.  Typically the kernel
clock is sourced from a hardware clock which isn't particularly
accurate in terms of frequency.  All clocks drift; cheap crystals on
computer boards drift a lot.  Ntpd will align both the frequency and
phase of the kernel clock using a PPS signal, but to compare the
various processing paths a PPS signal can go through, you must be able
to determine how much error came from the reference path and how much
from the path being tested.  In an ideal world, there would be no
measurement jitter, or frequency drift in the kernel clock, and thus
all PPS measurements made would be directly comparable to each other
without having to ascribe any part of the differences between sources
to the kernel clock.  

I am able to configure a Wandboard imx6 system so that the frequency
and phase alignment of kernel time is "perfect" with respect to one of
the PPS inputs.  Since all the PPS inputs are sourced by fanning out
the same source PPS signal, any difference in the offset or jitter
reported by ntpd directly represents differences in the processing
paths taken by those signals.  

The test setup consists of a commercial precision timing system which
generates a 10 MHz clock signal from a GPS-disciplined rubidium
oscillator, and it generates a PPS pulse that is derived from that 10
MHz clock using a simple "divide by 10 million" counter.  So the
leading edge of the PPS pulse is phase-coherent with the leading edge
of one of the clock pulses.  

The 10 MHz clock signal is fed to an external clock input pin on the
imx6 SoC.  Within the imx6 timer block, that clock signal drives a 32-
bit counter register, and that counter is used to implement a kernel
timecounter.  The PPS signal is also fed into that imx6 timer block,
and the leading edge of the PPS pulse causes the hardware to latch the
current value of the 32-bit timecounter into a capture register.  This
captured value is then used to generate a PPS measurement that doesn't
incorporate any interrupt processing latency or jitter.  

Because the PPS pulse and the kernel timecounter clock are derived from
the same source, the kernel clock will never appear to drift in
frequency.  That is, when ntpd compares two successive PPS
measurements, it will always find that exactly 1 billion nanoseconds
elapased between PPS pulses.  At startup, there will be some offset
between the kernel clock and the PPS pulse, and ntpd will slowly steer
out that error until the beginning of the kernel's second exactly
matches the PPS pulse edge.  

I force ntpd to step the clock at startup; that typically leaves about
a hundred microseconds of offset, and it takes a couple hours to slowly
reduce that to zero.  Once that happens, the system is in a steady
state, where ntpd will never again have to apply any correction to the
kernel clock.  You can see this state in the output of ntptime(8) which
reports on the kernel clock's phase and frequency.  The important
numbers are flagged with => 

  root # ntptime
  ntp_gettime() returns code 0 (OK)
    time e0eed6a9.b6875d58  Fri, Aug  2 2019 15:35:05.713, (.713003500),
    maximum error 8000 us, estimated error 1 us, TAI offset 0
  ntp_adjtime() returns code 0 (OK)
    modes 0x0 (),
=>  offset 0.000 us, frequency 0.000 ppm, interval 4 s,
    maximum error 8000 us, estimated error 1 us,
    status 0x2001 (PLL,NANO),
    time constant 4, precision 1.000 us, tolerance 496 ppm,
=>  pps frequency 0.000 ppm, stability 0.000 ppm, jitter 0.000 us,
    intervals 0, jitter exceeded 0, stability exceeded 0, errors 0.

Finally, for making these measurements, I wasn't worried about jitter
and latency that occurs down in the "handful of nanoseconds"
range.  For example, a single PPS source is fanned out into 4 different
inputs on the Wandboard, and I didn't worry at all about things like
using equal cable lengths, because I was looking for differences on the
order of microseconds, not nanos (and with a 10 MHz timecounter, the
measurement resolution is 100ns, so I can't see those little
differences anyway).  

-- Ian