[TEST/REVIEW] cpu time accounting patch, step 2

[Andrew Gallatin]

Poul-Henning Kamp writes:
 > Tests, reviews etc please.

Here are some benchmark results from my "AMD Athlon(tm) 64 X2 Dual
Core Processor 3800+" running an SMP kernel for both loopback and
10GbE TCP networks.  The executive summary is that there's a 41%
improvement in loopback pingpong, and a 12.5% improvement in a
pingpong test with a remote linux machine over 10GbE.  Since the linux
machine isn't infinately fast, the improvement is probably a bit more
than 12.5% ;)

The machine seemed stable, and I haven't seen anything negative
except for a tiny slowdown in sending using TCP_SENDFILE.

Thank you so much for doing this!!

Drew

Test data:

machdep.cpu_idle_hlt=0
net.isr.direct=1
dev.myri10ge.0.skip_pio_read=1

Everything else default except where noted for 10GbE TCP_RR  test.



netperf -tTCP_STREAM to localhost

x loopback_stream.before
+ loopback_stream.after
+--------------------------------------------------------------------------+
|x           x x x  x                                      +   + +     +  +|
|     |______A_M____|                                       |____M_A_____| |
+--------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5       4100.11       4221.45       4187.95      4178.016     46.300782
+   5       4472.37        4567.2          4510      4519.416     38.960833
Difference at 95.0% confidence
        341.4 +/- 62.4045
        8.17134% +/- 1.49364%
        (Student's t, pooled s = 42.7885)


netperf -tTCP_RR to localhost

x loopback_rr.before
+ loopback_rr.after
+--------------------------------------------------------------------------+
|x x                                                             +  ++   ++|
|MA|                                                              |__MA___||
+--------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5      27807.53      28195.85      27873.98     27964.662     163.75044
+   5      38590.71      40167.44      39301.22      39463.23      645.1375
Difference at 95.0% confidence
        11498.6 +/- 686.411
        41.1182% +/- 2.45457%
        (Student's t, pooled s = 470.647)


netperf -tTCP_STREAM to linux machine via 10GbE (128KB socket buffers)

x 10GbE_stream_rx.before
+ 10GbE_stream_rx.after
+--------------------------------------------------------------------------+
|                                   +                                      |
|x      x       x x   x             +                             + +    + |
|   |________A__M____|                |_________________A_________M_______||
+--------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5       7655.65       7723.43       7702.66      7693.382     27.071167
+   5       7768.29       7883.34       7863.13      7830.482     57.216535
Difference at 95.0% confidence
        137.1 +/- 65.2772
        1.78205% +/- 0.848484%
        (Student's t, pooled s = 44.7581)

x 10GbE_stream_tx.before
+ 10GbE_stream_tx.after
+--------------------------------------------------------------------------+
|+   x       x  +  +          x                       +        +  x       x|
|   |_|____________M__________MA_____A___________________|___________|     |
+--------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5       4640.64       4722.05       4670.05      4679.212     36.651045
+   5       4636.54       4709.22       4657.93      4671.276     31.004687
No difference proven at 95.0% confidence


netperf -tTCP_SENDFILE to linux machine via 10GbE (128KB socket buffers)

x 10GbE_sendfile_tx.before
+ 10GbE_sendfile_tx.after
+--------------------------------------------------------------------------+
| +  +     +               +            +       x            xxx          x|
||_________M_____A_______________|                  |_________A________|   |
+--------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5       7453.74       7523.76       7492.15      7490.258     24.844414
+   5       7328.59       7430.97       7355.29      7369.944     42.886543
Difference at 95.0% confidence
        -120.314 +/- 51.1132
        -1.60627% +/- 0.682395%
        (Student's t, pooled s = 35.0464)


netperf -tTCP_RR to linux machine via 10GbE, after setting nic interrupt
coalescing parameters to their minimum setting (dev.myri10ge.0.intr_coal_delay=1):

x 10GbE_rr.before
+ 10GbE_rr.after
+--------------------------------------------------------------------------+
|x  xx                                                             +  +++++|
||_AM|                                                               |_AM_||
+--------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x   5      29721.79      29929.62      29863.69      29827.82      89.80339
+   6      33323.05      33703.95      33614.14     33556.268     137.33838
Difference at 95.0% confidence
        3728.45 +/- 162.431
        12.4999% +/- 0.544562%
        (Student's t, pooled s = 118.588)