ZFS/compression/performance

[Freddie Cash]

On Wed, Oct 12, 2011 at 9:51 AM, Jeremy Chadwick
<freebsd at jdc.parodius.com>wrote:

> That might be the case on OpenSolaris but the performance hit on
> FreeBSD RELENG_8 is very high -- enough that enabling compression (using
> the defaults) causes stalls when I/O occurs (easily noticeable across
> SSH; characters are delayed/stalled (not buffered)), etc..
>
> The last time I tried it on RELENG_8 was right after ZFSv28 was MFC'd.
> If things have improved I can try again (I don't remember seeing any
> commits that could affect this), or if people really think changing the
> compression model to lzjb will help.
>

I would try it again, with the latest RELENG_8 sources.

compression=lzjb performance hit is negligible on our newest backups server
(8-core Opteron, 16 GB RAM, 4x 6-drive raidz2, dedupe enabled). I am able to
connect via SSH, navigate around the filesystem, tail -f multiple log files,
monitor network via iftop/trafshow, and watch top, all via a tmux session
(multiple tmux "windows").  This is with 5 rsync processes running,
transferring data at 200-300 Mbps.

compression=gzip-9 performance hit is noticeable on our older backups
servers (4-core Opteron, one has 8 GB RAM the other 12 GB RAM, 3x 8-drive
raidz2, no dedupe).  This pool is 93% full and heavily fragmented, though.
 SSH connections are slow, typing is choppy, basically everything is choppy,
while running 5 rsync processes.  This box used to run 12 rsyncs
simultaneously, but will now lockup completely if you try to run 7 or more.

I've switched to lzjb on these two boxes to see if that helps any.  I'm
thinking the fullness and fragmenting of the pool is the root cause of the
slowness on these boxes now.

-- 
Freddie Cash
fjwcash at gmail.com