numbers don't lie ...

Mike Meyer mwm-keyword-freebsdhackers2.e313df at
Wed Sep 13 12:08:35 PDT 2006

In <20060913172253.GA17499 at>, Kris Kennaway <kris at> typed:
> On Wed, Sep 13, 2006 at 01:00:25PM -0400, Mike Meyer wrote:
> > To illustrate, I have numbers for "make -j4" for a P4 with and without
> > hyperthreading enabled:
> > 
> > machdep.hyperthreading_allowed: 1 -> 0
> >      50m55.99s real      35m28s.19 user       8m20s.02 sys
> > machdep.hyperthreading_allowed: 0 -> 1
> >      38m48s.85 real      55m2s.43 user       12m27s.90 sys
> > 
> > Note the effect of the second CPU on the user time.
> i.e. since the hyperthreading virtual CPUs are not actually real CPUs,
> they spend a lot of time blocked in the same CPU core waiting for
> another hyperthread to release a resource, so the threads are both
> "running" from the point of view of the OS, but one is doing no work
> on the CPU a lot of the time.

In other words, hyperthreading makes the measurement FreeBSD takes to
see how much cpu time is being used nearly meaningless. I hadn't
realized that.

My understanding was that hyperthreading was intended to let the
system make more efficient use of the CPU, by providing two
instruction streams to be scheduled in the pipeline. This means you
get fewer bubbles in the pipeline, resulting in more work getting done
in the same number of cycles. The hyperthreads don't lock resources
per se, but there are lots of screwy rules about when things can be
put in the pipeline, leading to the same result - a hyperthread will
"wait" some number of steps in the pipeline for the rules to allow the
hyperthreads next step to happen. Later implementations of
hyperthreading relax the rules, meaning you get less waiting, or more
efficient use of the cpu, depending on how you want to look at it.

> This means that hyperthreading may or may not increase your
> performance depending on your workload (in your case it does).

Which is why I checked. This behavior isn't really different from any
other multi-CPU system: enabling another processor may or may not
increase your performance depending on your workload. In particular,
if some shared resource is the critical one for your workload and you
don't get more of it by turning on the second CPU, turning on the
second CPU will not improve performance, and will probably hurt it by
adding overhead. Hyperthreading is the worst case I know of, because
those CPUs share the CPU core.

I once benchmarked a VAX 8820 using a simulated multi-user
workload. The numbers said it supported more users with the second CPU
turned off. The reality was different, because my real workload
included long-running number-crunchers, where the simulated load
didn't. Those got serious benefit from the second CPU.

Mike Meyer <mwm at>
Independent Network/Unix/Perforce consultant, email for more information.

More information about the freebsd-hackers mailing list