svn commit: r279539 - head/sys/sys

[Davide Italiano]

On Mar 3, 2015 9:27 AM, "John-Mark Gurney" <jmg at funkthat.com> wrote:
>
> Julian Elischer wrote this message on Tue, Mar 03, 2015 at 01:20 -0800:
> > On 3/2/15 4:55 PM, Neel Natu wrote:
> > > Hi Davide,
> > >
> > > On Mon, Mar 2, 2015 at 12:26 PM, Davide Italiano <davide at freebsd.org>
wrote:
> > >> On Mon, Mar 2, 2015 at 12:05 PM, John-Mark Gurney <jmg at freebsd.org>
wrote:
> > >>> Author: jmg
> > >>> Date: Mon Mar  2 20:05:16 2015
> > >>> New Revision: 279539
> > >>> URL: https://svnweb.freebsd.org/changeset/base/279539
> > >>>
> > >>> Log:
> > >>>    give others fair warning that _SPARE2 isn't just cxgb, but used
by large
> > >>>    number of other subsystems, so you probably don't want _SPARE2..
> > >>>
> > >>>    ktr needs an overhaul to really only compile in the ones you
want,
> > >>>    we've long passed the 31 bits it provides..
> > >>>
> > >> If you really want to do the overhaul (which would be honestly
great),
> > >> I might consider revamping my work for per-cpu KTR buffer and include
> > >> that in the change. Originally it was just an exercise, but then it
> > >> evolved and I've been sitting with it in my local tree for a while. I
> > >> never had the chutzpah to upstream it because it involves fundamental
> > >> changes and breaks compatibility with the old ktrdump(1) format.
> > >> A rather outdated (and maybe not completely functional) version of
the
> > >> patch can be found here:
> > >> http://people.freebsd.org/~davide/locking/ktr_percpu.4.diff , which
> > >> should give you an high level view of the change.
> > >> I can update it to the last version and bring up for review, if
> > >> somebody think it might be a sane idea avoiding synchronization on a
> > >> single buffer for KTR.
> > I think it would be  a problem...
> > one of the truely useful things about ktr is that it does use a single
> > buffer.
> > this means that you get the true interaction between CPUS.
> > Schedgraph relies on this (as one example).
>
> Don't some systems provide a syncronized P-state invariant TSC?  If so,
> we can use the TSC clock to tell ordering between cores..
>
> I could definately seeing it be a tunable that lets people force either
> single buffer, or PCPU buffer KTR...  Where we know TSC is syncronized,
> we default to PCPU and others a single buffer...
>

I can't talk about schedgraph because I'm not familiar with the
implementation. Can you please elaborate how things will break with a
per-CPU buf?

I know that everything after Nehalem has a synchronized TSC.Also I've just
noticed Matt Dillon introduced a change similar to mine in Dragonfly about
10 years ago. The way they cope with TSC skew is that of resynchronizing
the timers periodically, e.g. 1 msec. This is exposed via a SYSCTL that can
be disabled on modern processors.
Anyhow, I tend to agree this kind of change might be kind of risky as is,
and I havent evaluated that on !IA32, which makes the proposal even more
problematic.

About the double implementation, I think it's not worth our time
duplicating the code + the burden of maintaining it. Either single or
per-CPU buffer. Given the initial opposition I'm inclined to leave the code
as is.

--
Davide