ZFS performance on 7.2-release/amd64 low compared to UFS2 + SoftUpdates

[Ivan Voras]

Dan Naumov wrote:
> I am wondering if the numbers I am seeing is something expected or is
> something broken somewhere. Output of bonnie -s 1024:

Unless you have 512 MB of memory in the machine or you're trying to test 
caching, the benchmark you did is useless.

In your environment, you need at least "-s 4096".

Even with those issues solved, it's semi-useless since you did both 
tests on the same drive, on different parts of it (see "diskinfo -vt 
ad0" or whatever your drive is to see how different parts of the drive 
have different performance). To make an objective comparison you need 
two identical drives, and create a new empty small-ish partition (e.g. 
15 GB) on the same position on both (e.g. at the start), then use this 
partition only for benchmarking (not for the OS, etc).

> on UFS2 + SoftUpdates:
> 
>               -------Sequential Output-------- ---Sequential Input-- --Random--
>               -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
> Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
>          1024 56431 94.5 88407 38.9 77357 53.3 64042 98.6 644511 98.6
> 23603.8 243.3
> 
> on ZFS:
> 
>               -------Sequential Output-------- ---Sequential Input-- --Random--
>               -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
> Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
>          1024 22591 53.7 45602 35.1 14770 13.2 45007 83.8 94595 28.0 102.2  1.2

I did my own testing on the early import of ZFS, the results in bonnie++ 
were that read and rewrite speeds are significantly better on ZFS than 
on UFS+SU (50%+), while write speed is a bit slower (~~10%).

There are of course other workloads than the sequential that need to be 
reviewed. For example, blogbench places ZFS again at about 50% better 
than UFS+SU, while randomio makes it 50% slower. Untarring the ports 
tree on ZFS is about 3x faster than on UFS+SU.