Benchmark: NetBSD 2.0 beats FreeBSD 5.3 in server performance

[Bosko Milekic]

On Thu, Jan 06, 2005 at 12:41:54PM +0100, Jesper Louis Andersen wrote:
> Quoting Bosko Milekic (bmilekic at technokratis.com):
> 
> > Not to mention, FreeBSD5 has yet to be micro-optimized.  How about some
> > scalability benchmarks on multi-CPU machines?  The original post
> > (particular since it was sent to -advocacy) is FUD.
> 
> How many CPU's do you have in mind? I would not expect FreeBSD to
> outperform NetBSD by much for a 2-CPU box with a typical server 
> workload with typical programs that does not even know to take advantage
> of a ''superior'' threading model. For a computer with 4-8 CPU's the
> advantage might be much bigger, but I have not yet seen any benchmarks 
> targetting computers with that number of CPUs. Partially because 
> people does not yet have access to such computers, partially because
> most people doesn't care about that kind of scalability.

  It's not true that people do not have access to such computers.

  Anyway, the point is that the benchmark results were pitched on our
  mailing lists as being an affirmative proof that NetBSD outperforms
  FreeBSD in various different setups which is, I maintain, complete
  FUD.  Measuring context switch time, micro-benchmark style, and
  concluding that this means that FreeBSD performs worse than NetBSD,
  period, is wrong.

  As for the multi-CPU case, it *is* important and it *is* relevant.  It
  might not be relevant if you're looking at userland-bound processes,
  but there *is* lots of kernel-bound processing that goes on in both
  end-host and forwarding setups, and it is unjust to compare and make
  sweeping conclusions regarding two systems, one which is designed to
  handle multi-processor scalability in-kernel, and one which is not.
  In some respect, it is like comparing Apples with Oranges.

> But this is speculation. I would like to see perfarmonce benchmarks for
> your scenario as well. 
> 
> I disagree that the original post is entirely FUD. While the conclusion
> is subjective, fact is that at the particular mix of microbenchmarks
> shows NetBSD faster than FreeBSD. I am wondering if that is the price
> you pay on single-cpu boxes to gain speed at the SMP boxes. And if this
> is true the question becomes if fine-grained locking is worth the
> implementation time when most computers are still single-cpu (Yes, I
> know this can change rapidly with the newer CPU types).

  It is also a question of development _stage_.  There are ways to
  further optimize the single CPU case, but attention has not
  necessarily been focused on that particular case yet, as significant
  architectural work remains (before further focus is diverted toward
  micro-optimizing).

> -- 
> jlouis

--
Bosko Milekic
bmilekic at technokratis.com
bmilekic at FreeBSD.org