svn commit: r325506 - in head: sbin/ifconfig sys/net sys/sys

[Konstantin Belousov]

On Tue, Nov 07, 2017 at 09:06:52AM -0800, John Baldwin wrote:
> On Tuesday, November 07, 2017 09:29:15 AM Konstantin Belousov wrote:
> > Author: kib
> > Date: Tue Nov  7 09:29:14 2017
> > New Revision: 325506
> > URL: https://svnweb.freebsd.org/changeset/base/325506
> > 
> > Log:
> >   Add a place for a driver to report rx timestamps in nanoseconds from
> >   boot for the received packets.
> >   
> >   The rcv_tstmp field overlaps the place of Ln header length indicators,
> >   not used by received packets.  The basic pkthdr rearrangement change
> >   in sys/mbuf.h was provided by gallatin.
> >   
> >   There are two accompanying M_ flags: M_TSTMP means that there is the
> >   timestamp (and it was generated by hardware).
> >   
> >   Another flag M_TSTMP_HPREC indicates that the timestamp is
> >   high-precision.  Practically M_TSTMP_HPREC means that hardware
> >   provided additional precision comparing with the stamps when the flag
> >   is not set.  E.g., for ConnectX all packets are stamped by hardware
> >   when PCIe transaction to write out the completion descriptor is
> >   performed, but PTP packet are stamped on port.  For Intel cards, when
> >   PTP assist is enabled, only PTP packets are stamped in the limited
> >   number of registers, so if Intel cards ever start support this
> >   mechanism, they would always set M_TSTMP | M_TSTMP_HPREC if hardware
> >   timestamp is present for the given packet.
> >   
> >   Add IFCAP_HWRXTSTMP interface capability to indicate the support for
> >   hardware rx timestamping, and ifconfig(8) command to toggle it.
> 
> Hmm, other NICs (Chelsio T4 and later for example) support timestamps that
> aren't in nanoseconds but some other frequency (which are themselves useful).
> It would be nice to have a more flexible interface that supports not only ns
> timestamps.  Perhaps a way to expose a direct hardware timestamp as a
> "number" without a specific frequency?

ConnectX does not provide ns-clocked counter either.  It is some internal
clock driven by a cristal with > 100MHz frequency.

There is no much space in the pkthdr, and the request to provide the
timestamp was in the context where the wall clock or some closely related
timer is needed.  Of course, I can put raw hardware timestamp into the
packet header, but only instead of the reduced value.  Then the consumer
of the timestamp would need to find the interface which received the
packet and call its method to convert ?  We have only one consumer in
tree (SO_TIMESTAMP) and perhaps one possible another consumer (TCP) for
this data, both of which require wall clock, so would need to call into
the method.

Also please see the discussion in the referenced review about accuracy of
the convertion.

Important example are Intel cards where is only limited number of
latched registers, and only PtP packets are stamped.  This (and some
quirk in ConnectX) explains the high-precision flag.