[CFR][CFT] counter(9): new API for faster and raceless counters
alfred at ixsystems.com
Wed Apr 3 18:00:56 UTC 2013
On 4/3/13 10:45 AM, Adrian Chadd wrote:
> On 3 April 2013 10:36, Alfred Perlstein <alfred at ixsystems.com> wrote:
>> Hey folks, sorry for the top post here, but I just came into this thread.
>> Here at iXsystems we've just developed a set of scripts to scrape the
>> various FreeBSD user land utilities (sysctl, netstat, nfsstat, vmstat, etc,
>> etc) and put them into graphs based on time.
>> The goal is to be able to line up all these metrics with whatever benchmark
>> we are currently running and be able to see what may be causing issues.
>> Potentially you should be able to scroll through the graphs and see things
>> like "ran out of mbufs @time", "vm system began paging at @time", "buffer
>> deaemon went nuts @time"
>> Then we can take the information back and leverage it to make tuning
>> decisions, or potentially change kernel algorithms.
>> The only problem we have is that every user land tool has its own format, so
>> along with my team we have written some shell to coerce the output from the
>> various programs into pseudo-CSV (key/value pair) which can then be post
>> processed by tools to convert to CSV which can then be put into something
>> like open office, or put through an R program to graph it.
>> I'm hoping to have something shortly.
>> What I was hoping to do over the next few days was discuss with people how
>> we can (or should we even) fix the user land statistics tools to output
>> machine readable output that can be easily parsed.
>> Example: netstat -m (hard to parse) versus 'vmstat -z | grep mbuf' easy to
>> The idea of outputting xml is good, CSV is OK, however CSV is problematic as
>> in the case of sysctl, if new nodes appear, then we can't begin to emit
>> them, we must either ignore them, or abort, or log them to auxiliary files.
>> Anything that makes life easier is good.
>> I should be able to share our scripts within the next couple of days.
> that's quite shiny.
> It'd be really nice if we could come up with a consistent statistics
> display, summary and export library.
> That way we could write tools that used a given data fetch/display API
> and then we could have optional "things" that implement various export
> For example, I'm packing up Sam Leffler's "libstatfoo" for inclusion
> into -HEAD, primarily so the tools that use it (wlanstats, mwlstats,
> athstats and all the other ath stats programs I'm using) can use it.
> But once I've converted the stats tool over to that, I can do a few cute things:
> * the library has a generic way to list all of the supported
> statistics fields - you register the statistic names with the library,
> then you can create arbitrary format strings with the information;
> * the library handles "now" versus "time series" data display itself -
> you just need to populate the statistic structure with the relevant
> stats and then call the right function to display things;
> * I plan on extending it to spit out CSV output as a generic feature,
> so I can start doing things like importing the output of those tools
> direct into rrdtool/etc without any intermediary parsing scripts;
> * .. and if I then add new stats fields, the (requested) output of the
> script won't change, so tools won't break.
One thing to note, the CSV format:
is vulnerable to problems where a new column will spring into being due
to loading of a kernel module/driver/something.
Imo it's better to look at XML or some other pseudo-CSV like:
so that we are OK with columns springing into existence or leaving.
More information about the freebsd-arch