small statistics tool in the tree ?

[Poul-Henning Kamp]

I written up a small statistics tool which is targeted at showing if
a benchmark proves anything or not.

critter phk> cat before
29195
29232
29226
29203
29211
critter phk> cat after
29179
29203
29195
29194
29196
critter phk> ./ministat before after
x before
+ after
+--------------------------------------------------------------------------+
|+                    +*+         *          x                    x       x|
|        |___________A_M___|_____|___________M__A_____________________|    |
+--------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5         29195         29232         29211       29213.4     15.469324
+   5         29179         29203         29195       29193.4     8.7920419
Difference at 95.0% confidence
        -20 ± 18.3497
        -0.0684617% ± 0.0628127%
        (Student's t, pooled s = 12.5817)

In this case the benchmark shows a very small improvement has obtained
with 95% probability

Here is a more convincing result:

x before
+ after
+--------------------------------------------------------------------------+
|                                                 +                        |
|                                                 +                        |
|x            x                                   +                        |
|x            x           x                       +                       +|
||_________A__M______|                      |_____M____A_________|         |
+--------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5         0.132         0.134         0.133        0.1328 0.00083666003
+   5         0.136         0.138         0.136        0.1364 0.00089442719
Difference at 99.5% confidence
        0.0036 ± 0.0024642
        2.71084% ± 1.85558%
        (Student's t, pooled s = 0.000866025)

The program is a <500 line .c program and I was wondering if it belongs
in the tree so we have an easy to use tool to point people at when they
run benchmarks.

	Source: http://phk.freebsd.dk/patch/ministat

Poul-Henning

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.