FreeBSD 5.3 I/O Performance / Linux 2.6.10

[Jesper Louis Andersen]

Quoting Nick Pavlica (linicks at gmail.com):

[Performance tests]

> Are there any good reasons for such a difference.  Your thoughts are
> appreciated.

There is so little information, so anything we throw your way will be
guesses. So I'll try to mention things one should be aware of when 
measuring performance and I/O.

First of all, check what you are comparing. Does FFS and XFS have the
same semantics? Is write-caching turned on? Is async-mounts being used,
etc. The old analogy of comparing apples to oranges still hold true,
sadly. 

Second, try to create a realistic workload instead of just counting 
what dd(1) can give you in raw read performance on the disk. The more
true your test are to your real workload, the better a measurement do
you get.

Third, the last 5% doesn't matter. System administration is about more
than just having the fastest operating system out there. Moore's law
might (not) be dead, but it doesn't take many months to win in 5% in
hardware. If you know linux well, then by all means use that, even if 
it is slowest. The rationale is that you know the system well and thus
avoid stability problems.

Fourth, stability. It is far more important to have a stable system
with an average load than an unstable system with low load.

Fifth, How much time are you spending on the benchmark? A weeks salary
could be used much better if poured into the machine. But alas, that
gives you no money.

Sixth, be _dead_ _sure_ that I/O is the performance bottleneck. One NIC
gone awry, and the bottleneck weren't where you expected it to be. I've
seen this happen too many times.

Seventh, locate the whole communication chain of your operating system
and hardware. For Disk I/O on FreeBSD that would mean disks,
disk-controllers, drivers, GEOM layer, UFS/FFS layer, VFS layer, ...,
application. If data are queried over the network, then the whole
network stack + additional drivers are touched too. One particular of
these might bottleneck. Locating it might mean a performance gain, but
it might be time-costly to do this.

Eight, use intuition to find interesting spots. Then use system query
tools to test the assertions the intuition made. Sometimes it is not
the hardware that needs a good cranking with the screwdriver, but the
software installation that could be told to behave better.

Have fun compressing the most out of your hardware.

-- 
jlouis