Jeremy Chadwick freebsd at jdc.parodius.com
Fri Oct 14 15:23:11 UTC 2011

On Wed, Oct 12, 2011 at 09:51:26AM -0700, Jeremy Chadwick wrote:
> On Wed, Oct 12, 2011 at 09:37:10AM -0700, David Brodbeck wrote:
> > On Wed, Oct 12, 2011 at 5:02 AM, Johannes Totz <jtotz at imperial.ac.uk> wrote:
> > 
> > > I just did a simple write test yesterday:
> > >
> > > 1) 6 MB/sec for gzip, 1.36x ratio
> > > 2) 34 MB/sec for lzjb, 1.23x ratio
> > >
> > > I'll stick with lzjb. It's good enough to get rid of most of the
> > > redundancy and speed is acceptable.
> > >
> > 
> > That's what we use on our text-heavy filesystems on our OpenSolaris server.
> >  (We work with large text corpora, so we have hundreds of gigabytes of pure
> > text.)  My benchmarks showed the performance hit for reads is nonexistent
> > when viewed over NFS, and the performance hit for writes is relatively
> > small...plus we don't write to that filesystem much.  We see about 1.5x
> > compression overall, with a little over 2x on some datasets that are
> > particularly compressible.
> That might be the case on OpenSolaris but the performance hit on
> FreeBSD RELENG_8 is very high -- enough that enabling compression (using
> the defaults) causes stalls when I/O occurs (easily noticeable across
> SSH; characters are delayed/stalled (not buffered)), etc..
> The last time I tried it on RELENG_8 was right after ZFSv28 was MFC'd.
> If things have improved I can try again (I don't remember seeing any
> commits that could affect this), or if people really think changing the
> compression model to lzjb will help.


Tried this out on RELENG_8 (2011/09/28) with ZFS v28 filesystems:

zfs create -o mountpoint=/test -o compression=lzjb data/test
cd /test
dd if=/dev/urandom of=testfile bs=16k

Then in another SSH session to the machine, held down a single key at my
bash prompt.  Every 3 seconds, like clockwork, SSH I/O would stall/drop
for about ~0.4 seconds.  CPU in the system is an Intel C2D E8400 (3GHz),
with ULE scheduler in use.

Then I did this:

rm testfile
zfs set compression=none data/test
dd if=/dev/urandom of=testfile bs=16k

And repeated the procedure: no stalls.

Then I tried using gzip-1: even worse.  The stalls were about 3-4 full
seconds long.  I imagine this is expected since gzip is much slower than

The important thing to note here is that the entire machine appears to
spin hard when ZFS compression is in use.  Even things like switching
virtual consoles (Alt-Fx) lock up until the compression bits do whatever
they need to do.

I will try to find a Solaris 10 test system at work later today and
tinker with compression there to see if it behaves the same, but given
what Bob described below I'm doubting it will:


Just an FYI for folks considering use of ZFS compression on RELENG_8 as
of this writing.

| Jeremy Chadwick                                jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                   Mountain View, CA, US |
| Making life hard for others since 1977.               PGP 4BD6C0CB |

More information about the freebsd-fs mailing list