Some filesystem performance numbers

[Garrett Wollman]

I recently bought a copy of the SPECsfs2014 benchmark, and I've been
using it to test out our NFS server platform.  One scenario of
interest to me is identifying where the limits are in terms of the
local CAM/storage/filesystem implementation versus bottlenecks unique
to the NFS server, and to that end I've been running the benchmark
suite directly on the server's local disk.  (This is of course also
the way you'd benchmark for shared-nothing container-based
virtualization.)

I have found a few interesting results on my test platform:

1) I can quantify the cost of using SHA256 vs. fletcher4 as the ZFS
checksum algorithm.  On the VDA workload (essentially a simulated
video streaming/recording application), my server can do about half as
many "streams" with SHA256 as it can with fletcher4.

2) Both L2ARC and separate ZIL have small but measurable performance
impacts.  I haven't examined the differences closely.

3) LZ4 compression also makes a small performance impact, but as
advertised, less than LZJB for mostly-incompressible data.

I hope to be able to present the actual benchmark results at some
point, as well as some results for the other three workloads.

-GAWollman