ZFS: How to enable cache and logs.

[Alexander Leidinger]

Quoting Danny Carroll <fbsd at dannysplace.net> (from Thu, 12 May 2011  
12:26:36 +1000):

> Replying to myself in order to summarise the recommendations (when using
> v28):
>  - Don't use SSD for the Log device.  Write speed tends to be a problem.

It depends. You could buy a lot of large and low-power (and sort of  
slow) disks for the raw storage space, and 2 fast but small disks for  
the log. Even if they are not SSDs, this could improve the write  
throughput of the system (also depends upon the bus bandwith of the  
system, the disk-controller (SATA/SCSI) and your workload).

The important part is that normally the log devices should have a  
lower latency and faster transfer speed than the main pool to be  
effective (you may get an improvement even if the devices have the  
same specs, as the main pool does not see the same workload then, but  
it depends upon your workload).

>  - SSD ok for cache if the sizing is right, but without TRIM, don't
> expect to take full advantage of the SSD.

As long as we do not have TRIM to play with, we can not give a proper  
answer here. For sure we can tell that a SSD increases the max  
performance a pool can deliver by a good amount.

I would expect that TRIM can give some improvement for a cache device,  
but I do not expect that it is much. If it is more than 10% I would be  
very surprised. I expect the improvement more in the <=5% range for  
the cache (which may make a difference in read-cases where you are  
hitting the limit).

>  - Do use two devices for log and mirror them with ZFS.  Bad things
> *can* happen if*all* the log devices die.

s/can/will/ as you will lose data in this case. The difference between  
v15 and v28 is the amount of data you lose (the entire pool vs. only  
what is still on the log devices)

>  - Don't colocate L2ARC and Log devices.

This depends upon the devices and your workload. If you do not have a  
lot of throughput, but your applications has some hard requirements  
regarding the latency, it may make sense.

Without measuring the outcome on your own workload, there is not  
really a way to answer this, but if your workload is read and write  
limited, to add first a separate log device. This way the pool is  
freed of the sync-writes, the read performance should increase and the  
write performance too as the data goes to the log device first without  
interfering with reads (in this case it matters more that the device  
is a separate device than that the device is significantly faster).  
Only when this is done and there is more demand regarding reads, I  
would add a significantly faster cache device (or more RAM, depending  
on the specs of the machine and the workload).

Another of disk-tuning: if you are doing something like this on your  
workstation / development system where you just want to have the  
comfort of not waiting for the disks (and the workload does not demand  
a lot of reads+writes at the same time), you could give a shared  
cache/log device a try (but the device needs to be significantly  
faster to feel a difference).

> Remaining questions.
> - Is there any advantage to using a spare partition on a SCSI or SATA
> drive as L2Arc?  Assuming it was in the machine already but doing nothing?

It depends. In general: if the cache is faster (by an amount which  
matters to you) than the pool, it helps. So for your particular case:  
if the other partitions on such a drive are only used occasionally and  
the drive is faster than your pool, you could get a little bit more  
out of it if you add the unused partition as a cache.

As like for all RAIDs, more spindles (disks) give better performance,  
as such it could help if the cache has the same characteristics as the  
rest of the pool, but this depends upon your workload. In such a case  
it is probably better to add the disks to the pool instead of using it  
as a cache. A definitive answer to this can only be obtained by  
running your workload on both setups and compare the results (zpool  
iostat / gstat).

Bye,
Alexander.

-- 
There are three kinds of people: men, women, and unix.

http://www.Leidinger.net    Alexander @ Leidinger.net: PGP ID = B0063FE7
http://www.FreeBSD.org       netchild @ FreeBSD.org  : PGP ID = 72077137