ZFS cpu requirements, with/out compression and/or dedup
Bob Friesenhahn
bfriesen at simple.dallas.tx.us
Sat Sep 19 14:08:38 UTC 2015
On Sat, 19 Sep 2015, Sami Halabi wrote:
> Hi everyone,
> I've been searching the documentation, wiki.. etc. but found no rule of
> thumb to CPU requirements to run ZFS fs. for memory there are few rules
> according to using dedup or not.
>
> rules of thumb I concluded are so far:
> 1. use compression lz4.
> 2. use checksum.
> 3. disable atime.
>
> from what i read the status of dedup is not that clear and seems there are
> bugs and better to avoid it?
I am not so sure about dedup bugs. Usually the problem encountered
with dedup is unrealistic expectations about how it will behave.
Dedup requires a large amount of RAM and/or L2ARC device which are
appropriately sized for the amount of storage and the amount of
redundancy encountered. The problem typically encountered is that
things work "fine" with dedup until one day they do not and when they
do not, it is like going off a cliff. Don't enable dedup unless you
have carefully researched it, decided that its value outweighs its
cost, and make sure that you have enough RAM or SSD-based L2ARC
(preferably RAM) so that the tables it needs are all readily available
without accessing rotating media.
Depending on what you were planning to use dedup for, there may be
other zfs tricks (e.g. cloning) which are amost as effective, but
without the risks.
> so according to 1-3 above what cpu requirements i need? is ATOM cpu like
> supermicro c2750/3/5/8 enough to run system of 20TB /40TB with 1-3 above?
> if dedup IS enabled would it still work fine?
CPU usage is rarely a problem for zfs except for when compression is
involved. Lz4 is supposed to be CPU-efficient on Intel CPUs. Don't
use gzip compression if you don't have a powerful CPU. Compression
can be changed at any time, although a given file will remain
compressed with the compression setting in effect when it was created.
CPU use is driven by I/O requirements and selected compression
algorithms and not by the size of the storage pool.
Bob
--
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
More information about the freebsd-fs
mailing list