freebsd-fs Digest, Vol 437, Issue 3
Radio młodych bandytów
radiomlodychbandytow at o2.pl
Wed Nov 2 12:45:32 UTC 2011
On 2011-11-02 13:00, freebsd-fs-request at freebsd.org wrote:
> Message: 1
> Date: Tue, 1 Nov 2011 22:55:36 +0530
> From: Shivaram Upadhyayula<shivaram.u at quadstor.com>
> Subject: Re: ZFS/compression/performance
> To: Dennis Glatting<freebsd at penx.com>
> Cc:freebsd-fs at freebsd.org
> <CAN-_EfxVZchM-ByjbtMO2kcjiU_=u7GF5q5_9rz7HHeL09Vn0g at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
> On Thu, Oct 13, 2011 at 9:47 AM, Dennis Glatting<freebsd at penx.com> wrote:
>> > On Wed, 2011-10-12 at 08:59 +0100, Steven Hartland wrote:
>> > ----- Original Message -----
>>> >> From: "Dennis Glatting"<freebsd at penx.com>
>>> >> Have you tried using the alternative compression algorithms
>>> >> e.g. lzjb or gzip-[1-5] the default gzip = gzip-6
>> > I have tried lzjb and I am unimpressed. I have not tried different levels of
>> > gzip on ZFS but I have tried it on documents with results I expected.
>> > As I mentioned, I have a lot of data. Two files were 26GB uncompressed but I
>> > had to kill those data sets because I ran out of room (I have reorganized my
>> > arrays since then). My ZFS compression ratio is 4.93x and I would require
>> > more storage at different gzip levels or ljzb.
>> > An option is not too compress with ZFS rather directly with gzip however I
>> > would still need lots of temporary storage for manipulation, which is what I
>> > am doing now (e.g., sort). Processing with zcat isn't always a good solution
>> > because some applications want files, but you have to do what you have to
>> > do.
> A few years back there was a discussion of about the possibility of
> other compression algorithms in ZFS
> But it looks like there hasn't been anything much further on that. I
> have recently started using ZFS and during that time i have tried out
> ZFS with lzf (http://oldhome.schmorp.de/marc/liblzf.html) and it seems
> to perform much better, both in speed and ratio over lzjb.
> Anyway my point is that, somewhere down the line other compression
> algorithms should be evaluated. gzip seems slow and it looks like lzjb
> may not be sufficient. For anyone interested, I have attached some of
> the tests i had run and the diff for lzf support.
> -- Reduce Storage expenditure with QUADStor Storage Virtualization
I did some synthetic tests and lzf was OK, but not great. LZJB was
I'm in the process of doing a more comprehensive part 3 and I think that
LZ4 should be better.
More information about the freebsd-fs