[PATCH] pmcannotate tool

[Attilio Rao]

pmcannotate is a tool that prints out sources of a tool (in C or
assembly) with inlined profiling informations retrieved by a prior
pmcstat analysis.
If compared with things like callgraph generation, it prints out
profiling on a per-instance basis and this can be useful to find, for
example, badly handled caches, too high latency instructions, etc.

The tool usage is pretty simple:
pmcannotate [-a] [-h] [-k path] [-l level] samples.out binaryobj

where samples.out is a pmcstat raw output and binaryobj is the binary
object that has been profiled and is accessible for (ELF) symbols
retrieving.
The options are better described in manpages but briefly:
- a: performs analysis on the assembly rather than the C source
- h: usage and informations
- k: specify a path for the kernel in order to locate correct objects for it
- l: specify a lower boundary (in total percentage time) after which
functions will be displayed nomore.

A typical usage of pmcannotate can be some way of kernel annotation.
For example, you can follow the steps below:
1) Generate a pmc raw output of system samples:
# pmcstat -S ipm-unhalted-core-cycles -O samples.out
2) Copy the samples in the kernel building dir and cd there
# cp samples.out /usr/src/sys/i386/compile/GENERIC/ ; cd
/usr/src/sys/i386/compile/GENERIC/
3) Run pmcannotate
# pmcannotate -k . samples.out kernel.debug > kernel.ann

In the example above please note that kernel.debug has to be used in
order to produce a C annotated source. This happens because in order
to get the binary sources we rely on the "objdump -S" command which
wants binary compiled with debugging options.
If not debugging options are present assembly analynsis is still
possible, but no C-backed one will be available.
objdump is not the only one tool on which pmcannotare rely. Infact, in
order to have it working, pmcstat needs to be present too because we
need to retrieve, from the pmcstat raw output, informations about the
sampled PCs (in particular the name of the function they live within,
its start and ending addresses). As long as currently pmcstat doesn't
return those informations, a new option has been added to the tool
(-m) which can extract (from a raw pmcstat output) all pc sampled,
name of the functions and symbol bundaries they live within.

Also please note that pmcannotate suffers of 2 limitations.
Firstly, relying on objdump to dump the C source, with heavy
optimization levels and lots of inlines the code gets difficult to
read. Secondly, in particular on x86 but I guess it is not the only
one case, the sample is always attributed to the instruction directly
following the one that was interrupted. So in a C source view some
samples may be attributed to the line below the one you're interested
in.  It's also important to keep in mind that if a line is a jump
target or the start of a function the sample really belongs elsewhere.

The patch can be found here:
http://www.freebsd.org/~attilio/pmcannotate.diff/

where pmcannotate/ dir contains the code and needs to go under
/usr/src/usr.sbin/ and the patch has diffs against pmcstat and
Makefile.

This work has been developed on the behalf of Nokia with important
feedbacks and directions from Jeff Roberson.

Testing and feedbacks (before it hits the tree) are welcome.

Thanks,
Attilio


-- 
Peace can only be achieved by understanding - A. Einstein