HZ=100: not necessarily better?

Robert Watson rwatson at FreeBSD.org
Sat Jun 17 12:50:49 UTC 2006

Scott asked me if I could take a look at the impact of changing HZ for some 
simple TCP performance tests.  I ran the first couple, and got some results 
that were surprising, so I thought I'd post about them and ask people who are 
interested if they could do some investigation also.  The short of it is that 
we had speculated that the increased CPU overhead of a higher HZ would be 
significant when it came to performance measurement, but in fact, I measure 
improved performance under high HTTP load with a higher HZ.  This was, of 
course, the reason we first looked at increasing HZ: improving timer 
granularity helps improve the performance of network protocols, such as TCP. 
Recent popular opinion has swung in the opposite direction, that higher HZ 
overhead outweighs this benefit, and I think we should be cautious and do a 
lot more investigating before assuming that is true.

Simple performance results below.  Two boxes on a gig-e network with if_em 
ethernet cards, one running a simple web server hosting 100 byte pages, and 
the other downloading them in parallel (netrate/http and netrate/httpd).  The 
performance difference is marginal, but at least in the SMP case, likely more 
than a measurement error or cache alignment fluke.  Results are 
transactions/second sustained over a 30 second test -- bigger is better; box 
is a dual xeon p4 with HTT; 'vendor.*' are the default 7-CURRENT HZ setting 
(1000) and 'hz.*' are the HZ=100 versions of the same kernels.  Regardless, 
there wasn't an obvious performance improvement by reducing HZ from 1000 to 
100.  Results may vary, use only as directed.

What we might want to explore is using a programmable timer to set up high 
precision timeouts, such as TCP timers, while keeping base statistics 
profiling and context switching at 100hz.  I think phk has previously proposed 
doing this with the HPET timer.

I'll run some more diverse tests today, such as raw bandwidth tests, pps on 
UDP, and so on, and see where things sit.  The reduced overhead should be 
measurable in cases where the test is CPU-bound and there's no clear benefit 
to more accurate timing, such as with TCP, but it would be good to confirm 

Robert N M Watson
Computer Laboratory
University of Cambridge

peppercorn:~/tmp/netperf/hz> ministat *SMP
x hz.SMP
+ vendor.SMP
|xx x xx   x       xx  x     +              +   +  +   +    ++ +         ++|
|  |_______A________|                     |_____________A___M________|     |
     N           Min           Max        Median           Avg        Stddev
x  10         13715         13793         13750       13751.1     29.319883
+  10         13813         13970         13921       13906.5     47.551726
Difference at 95.0% confidence
         155.4 +/- 37.1159
         1.13009% +/- 0.269913%
         (Student's t, pooled s = 39.502)

peppercorn:~/tmp/netperf/hz> ministat *UP
x hz.UP
+ vendor.UP
|x           x xx   x      xx+   ++x+   ++  * +    +                      +|
|         |_________M_A_______|___|______M_A____________|                  |
     N           Min           Max        Median           Avg        Stddev
x  10         14067         14178         14116       14121.2     31.279386
+  10         14141         14257         14170       14175.9     33.248058
Difference at 95.0% confidence
         54.7 +/- 30.329
         0.387361% +/- 0.214776%
         (Student's t, pooled s = 32.2787)

More information about the freebsd-performance mailing list