ZFS Performance FreeBSD 9.0 vs. Openindiana
Franz Schober
franz.schober at firmos.at
Thu Mar 29 09:09:04 UTC 2012
Hi,
I am testing a new storage hardware (Xeon E5606, 24 GB Ram, LSI1068
controller in non-raid mode, 8 7200 SATA Disks, 2 OCZ SSD ZIL Mirror),
for a NFS/iSCSI storage system.
Currently I'm doing basic performance tests of the ZFS filesystem. We
are using FreeBSD for a couple of years for various services in our
software development company, so FreeBSD is our first choice.
For performance tests, I use the iozone benchmark with a multi-streaming
concurrency test,
iozone -o -c -t 8 -r 128k -s 4G (Sync Mode, 8 concurrent workers, 128 k
Recordsize, 4G working file for every worker to run not only in cache).
ZPool ist a stripe of the 8 SATA Disks and a mirror of the two SSD Log
disks (here the FreeBSD zpool status):
pool: pools
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
pools ONLINE 0 0 0
da0 ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
da3 ONLINE 0 0 0
da4 ONLINE 0 0 0
da5 ONLINE 0 0 0
da6 ONLINE 0 0 0
da7 ONLINE 0 0 0
logs
mirror-8 ONLINE 0 0 0
ada0 ONLINE 0 0 0
ada1 ONLINE 0 0 0
Comparing FreeBSD 9.0-RELEASE (amd64) to OpenIndiana 151a, I get a
performance difference, where FreeBSD is slower in most tests,
the pools is freshly created before each test:
FreeBSD:
Iozone: Performance Test of File I/O
Version $Revision: 3.397 $
Compiled for 64 bit mode.
Build: freebsd
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave
Boone,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer.
Ben England.
Run began: Thu Mar 29 10:39:47 2012
SYNC Mode.
Include close in write timing
Record Size 128 KB
File size set to 4194304 KB
Command line used: iozone -o -c -t 8 -r 128k -s 4G
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 8 processes
Each process writes a 4194304 Kbyte file in 128 Kbyte records
Children see throughput for 8 initial writers = 150780.28 KB/sec
Parent sees throughput for 8 initial writers = 150767.08 KB/sec
Min throughput per process = 18846.89 KB/sec
Max throughput per process = 18849.44 KB/sec
Avg throughput per process = 18847.53 KB/sec
Min xfer = 4193792.00 KB
Children see throughput for 8 rewriters = 150051.28 KB/sec
Parent sees throughput for 8 rewriters = 150048.26 KB/sec
Min throughput per process = 18615.49 KB/sec
Max throughput per process = 18837.79 KB/sec
Avg throughput per process = 18756.41 KB/sec
Min xfer = 4144896.00 KB
Children see throughput for 8 readers = 1001472.78 KB/sec
Parent sees throughput for 8 readers = 1001417.37 KB/sec
Min throughput per process = 118845.12 KB/sec
Max throughput per process = 138027.58 KB/sec
Avg throughput per process = 125184.10 KB/sec
Min xfer = 3611520.00 KB
Children see throughput for 8 re-readers = 876152.78 KB/sec
Parent sees throughput for 8 re-readers = 870610.71 KB/sec
Min throughput per process = 106323.10 KB/sec
Max throughput per process = 114136.91 KB/sec
Avg throughput per process = 109519.10 KB/sec
Min xfer = 3909120.00 KB
Children see throughput for 8 reverse readers = 1148130.64 KB/sec
Parent sees throughput for 8 reverse readers = 1142916.22 KB/sec
Min throughput per process = 126814.09 KB/sec
Max throughput per process = 172099.73 KB/sec
Avg throughput per process = 143516.33 KB/sec
Min xfer = 3117440.00 KB
Children see throughput for 8 stride readers = 299141.80 KB/sec
Parent sees throughput for 8 stride readers = 298399.38 KB/sec
Min throughput per process = 29238.70 KB/sec
Max throughput per process = 45995.27 KB/sec
Avg throughput per process = 37392.73 KB/sec
Min xfer = 2667264.00 KB
Children see throughput for 8 random readers = 115979.46 KB/sec
Parent sees throughput for 8 random readers = 115974.77 KB/sec
Min throughput per process = 14312.92 KB/sec
Max throughput per process = 14753.71 KB/sec
Avg throughput per process = 14497.43 KB/sec
Min xfer = 4068992.00 KB
Children see throughput for 8 mixed workload = 188689.19 KB/sec
Parent sees throughput for 8 mixed workload = 188673.96 KB/sec
Min throughput per process = 12175.37 KB/sec
Max throughput per process = 34954.12 KB/sec
Avg throughput per process = 23586.15 KB/sec
Min xfer = 1460992.00 KB
Children see throughput for 8 random writers = 147947.42 KB/sec
Parent sees throughput for 8 random writers = 147939.94 KB/sec
Min throughput per process = 18492.69 KB/sec
Max throughput per process = 18494.70 KB/sec
Avg throughput per process = 18493.43 KB/sec
Min xfer = 4193920.00 KB
Children see throughput for 8 pwrite writers = 149780.00 KB/sec
Parent sees throughput for 8 pwrite writers = 149772.91 KB/sec
Min throughput per process = 18721.60 KB/sec
Max throughput per process = 18723.81 KB/sec
Avg throughput per process = 18722.50 KB/sec
Min xfer = 4193792.00 KB
Children see throughput for 8 pread readers = 1014824.33 KB/sec
Parent sees throughput for 8 pread readers = 1014743.46 KB/sec
Min throughput per process = 114311.04 KB/sec
Max throughput per process = 150336.52 KB/sec
Avg throughput per process = 126853.04 KB/sec
Min xfer = 3189376.00 KB
iozone test complete.
OpenIndiana:
Iozone: Performance Test of File I/O
Version $Revision: 3.323 $
Compiled for 32 bit mode.
Build: Solaris10cc-64
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root.
Run began: Fri Mar 16 14:11:48 2012
SYNC Mode.
Include close in write timing
Record Size 128 KB
File size set to 4194304 KB
Command line used: /usr/benchmarks/iozone/iozone -o -c -t 8 -r 128k
-s 4G
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 8 processes
Each process writes a 4194304 Kbyte file in 128 Kbyte records
Children see throughput for 8 initial writers = 163315.97 KB/sec
Parent sees throughput for 8 initial writers = 163238.81 KB/sec
Min throughput per process = 20399.37 KB/sec
Max throughput per process = 20423.74 KB/sec
Avg throughput per process = 20414.50 KB/sec
Min xfer = 4189440.00 KB
Children see throughput for 8 rewriters = 163415.44 KB/sec
Parent sees throughput for 8 rewriters = 163298.96 KB/sec
Min throughput per process = 20266.25 KB/sec
Max throughput per process = 20611.95 KB/sec
Avg throughput per process = 20426.93 KB/sec
Min xfer = 4123904.00 KB
Children see throughput for 8 readers = 1180533.94 KB/sec
Parent sees throughput for 8 readers = 1180090.09 KB/sec
Min throughput per process = 121104.02 KB/sec
Max throughput per process = 198012.62 KB/sec
Avg throughput per process = 147566.74 KB/sec
Min xfer = 2565376.00 KB
Children see throughput for 8 re-readers = 951965.89 KB/sec
Parent sees throughput for 8 re-readers = 948751.85 KB/sec
Min throughput per process = 118049.84 KB/sec
Max throughput per process = 120618.28 KB/sec
Avg throughput per process = 118995.74 KB/sec
Min xfer = 4113536.00 KB
Children see throughput for 8 reverse readers = 3024070.31 KB/sec
Parent sees throughput for 8 reverse readers = 2867010.77 KB/sec
Min throughput per process = 318376.59 KB/sec
Max throughput per process = 557625.75 KB/sec
Avg throughput per process = 378008.79 KB/sec
Min xfer = 2563968.00 KB
Children see throughput for 8 stride readers = 335398.05 KB/sec
Parent sees throughput for 8 stride readers = 331540.81 KB/sec
Min throughput per process = 32520.95 KB/sec
Max throughput per process = 62529.34 KB/sec
Avg throughput per process = 41924.76 KB/sec
Min xfer = 2210048.00 KB
Children see throughput for 8 random readers = 120422.72 KB/sec
Parent sees throughput for 8 random readers = 120418.39 KB/sec
Min throughput per process = 14456.03 KB/sec
Max throughput per process = 15568.85 KB/sec
Avg throughput per process = 15052.84 KB/sec
Min xfer = 3894528.00 KB
Children see throughput for 8 mixed workload = 191391.66 KB/sec
Parent sees throughput for 8 mixed workload = 191249.48 KB/sec
Min throughput per process = 12364.50 KB/sec
Max throughput per process = 35442.05 KB/sec
Avg throughput per process = 23923.96 KB/sec
Min xfer = 1463296.00 KB
Children see throughput for 8 random writers = 163556.06 KB/sec
Parent sees throughput for 8 random writers = 163218.36 KB/sec
Min throughput per process = 20411.16 KB/sec
Max throughput per process = 20484.72 KB/sec
Avg throughput per process = 20444.51 KB/sec
Min xfer = 4179328.00 KB
iozone test complete.
Now my question: Which parameters in the ZFS Subsystem of FreeBSD are
tuneable to reach the same performance in FreeBSD,
especially pushing the synchronous write performance ?
Thx a lot,
Franz Schober
More information about the freebsd-fs
mailing list