Some ZFS+NFS benchmarks (OpenSolaris)

[Bernd Walter]

On Wed, Feb 24, 2010 at 10:24:33AM +0100, Alexander Leidinger wrote:
> 
> Quoting Bernd Walter <ticso at cicely7.cicely.de> (from Wed, 24 Feb 2010  
> 00:08:44 +0100):
> 
> >Yes - your values seem to fit with my assumed values, but I'm talking
> >about L2ARC cache devices here.
> >Those are written linear with small bandwith and read random.
> >If they are too slow ZFS just drops data and cache fill slower.
> 
> The cache is filled asynchron. And there is also no purge from ARC to  
> L2ARC, so no increased latency if something needs to be thrown out of  
> the ARC. The write speed does not really matter for L2ARC devices.
> 
> >Random read access for USB sticks is great compared to HDD - although
> >USB has a high latency overhead.
> 
> It depends, the corresponding numbers I present in
> http://www.leidinger.net/blog/2010/02/10/making-zfs-faster/
> are mostly taken with gstat and depending on the workload I see only  
> 0.4ms per read on the L2ARC.

This matches with my experience, which is already impressive fast for
USB as such.
But under load the latency only increses to a few ms, while HDD easily
climb up to a few hundred ms.
Even half rotational delay without seek time for a 7200 HDD is about
4ms - 10 times slower than USB stick.
With USB sticks there is only a high latency expection from time to
time when there was also write load, but this rarely happens for me,
especially after adding multiple devices.
What I want to say is that Performance wise SSD are the better choice,
but depending on the workload the higher capacity you get with USB sticks
can be more important.
I'm currently running a mix of 1x 4G CF, 4x 4G USB, 1x 32G SATA SSD.
Speed with SSD is best, read latency with CF isn't much different from
USB, but price is higher.
I've ordered 4x 16G cheap sticks, since I'm very pleased with the speed,
but want a higher hit rate.
The only drawback with USB sticks is that my BIOS sees them and is
horrible slow to boot.

-- 
B.Walter <bernd at bwct.de> http://www.bwct.de
Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.