ZFS performance help sought

Mason Loring Bliss mason at blisses.org
Thu Jan 21 20:51:48 UTC 2016


Hi all.

I've bounced back and forth between FreeBSD and Linux, and one of the reasons
why I tend to part with FreeBSD is frustration with ZFS performance. I'm
using my desktop for a couple roles at home. One of the roles is that it
aggregates back-ups from across my machines and gathers them onto a back-up
pool.

Running FreeBSD, a zfs send/receive from one pool to another makes my system
almost unusably slow and even begins to dig me into swap a little. As a test
a little while back, I capped the ARC at half RAM, but that didn't matter a
bit. I started looking into scheduler tweaking when I decided to take the
path of least resistance and just install Linux instead. ZFS on Linux on
literally the same hardware, dealing with the same pools, handles this same
disk I/O without a hiccough.

I've moved back to FreeBSD on this box now, and I'd like to resolve this
issue. I don't know if it's a matter of fixing something broken in FreeBSD's
scheduling or tuning ZFS somehow such that it's friendlier. (For what it's
worth, renice'd zfs processes don't make a bit of difference, just as capping
ARC didn't.)

The box has FreeBSD 10.2, eight gigs of RAM, and I'm dealing with pools 1TB
or smaller. No deduplication. I'm not enough of a ZFS guru to have a strong
notion of what needs to change. I've not seen anything that seems
particularly relevant in tuning guides. I have precious little diagnostic
data to share. That said, here's a quick idea of what FreeBSD is doing,
captured last night, with my box doing precious little else beyond the
transfer:

last pid:  2631;  load averages:  9.44,  8.94,  7.28    up 0+00:29:03  22:04:12
58 processes:  2 running, 56 sleeping
CPU:  0.8% user,  0.0% nice, 95.8% system,  0.1% interrupt,  3.3% idle
Mem: 136M Active, 14M Inact, 6915M Wired, 8784K Cache, 855M Free
ARC: 6234M Total, 248M MFU, 5233M MRU, 626M Anon, 45M Header, 82M Other
Swap: 8192M Total, 884K Used, 8191M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
 1334 root          1  52   15 42248K  3164K RUN     0   1:27   7.57% zfs
 1333 root          1  38   15 42248K  3232K pipewr  1   1:04   4.98% zfs

Not that this matters in grand scheme of things, but I'm hoping to get a
handle on what's happening here before frustration drives me back to Linux.
I'd be happy to gather diagnostics given some pointers on what would be
useful. It seems unlikely that FreeBSD is this desperately inferior to Linux
in terms of the competency of its scheduler, but I'm not sure what to tune to
bring it up to the generally usable state I see on the same hardware under
Linux.

Thanks in advance!

-- 
Mason Loring Bliss   ((  "In the drowsy dark cave of the mind dreams
mason at blisses.org     ))  build  their nest  with fragments  dropped
http://blisses.org/  ((   from day's caravan." - Rabindranath Tagore


More information about the freebsd-questions mailing list