some ZFS questions

[Paul Kraus]

On Aug 26, 2014, at 13:33, Daniel Staal <DStaal at usa.net> wrote:

> Compression can be well worth the CPU cost, actually: In many cases using light/medium compression under ZFS *improves* performance, as it takes less time to compress/decompress the data then it does to transfer the uncompressed data to disk.

The issue comes down to three things:

1) How compressible is *your* data? Scott said that his data is already compressed, so that probably means that ZFS compression will not achieve much.

2) How much excess CPU capacity do you have and how fast is it? If your CPUs are idle 80% of the time and they are fast, then they may decrease the amount of data to be written by enough to make it worthwhile.

3) How fast is your storage? If your storage system is faster than your *excess* CPU capacity, then compression may not get you any performance boost.

You are balancing the amount by which the compression will reduce the amount of data to be written (and later read) by the additional CPU overhead involved in the compression. Also remember that the speed of your storage will probably be different for writes vs. reads. If your data access is 95% read, then optimizing writes through compression may not be the best answer.

> Experiment with your data, of course; it depends on how compressible things are, and your hardware.  But don't write it off just because it's a CPU cost - it might still be a good option.

Excellent advice. If I can, I test the compressibility of my data sets, often with unexpected results. For example, I discovered that raw audio files (WAV or AIFF) do NOT compress very much using the default ZFS compression, so I do not enable compression on datasets where those files reside. On the other hand, I get between 2:1 and 3:1 compression on virtual machine disk images (even static ones), so I enable compression on those datasets.

One more argument for separate datasets based on the data type.

--
Paul Kraus
paul at kraus-haus.org