ZFS performance help sought

dweimer dweimer at dweimer.net
Thu Jan 21 21:35:33 UTC 2016

On 2016-01-21 2:51 pm, Mason Loring Bliss wrote:
> Hi all.
> I've bounced back and forth between FreeBSD and Linux, and one of the 
> reasons
> why I tend to part with FreeBSD is frustration with ZFS performance. 
> I'm
> using my desktop for a couple roles at home. One of the roles is that 
> it
> aggregates back-ups from across my machines and gathers them onto a 
> back-up
> pool.
> Running FreeBSD, a zfs send/receive from one pool to another makes my 
> system
> almost unusably slow and even begins to dig me into swap a little. As a 
> test
> a little while back, I capped the ARC at half RAM, but that didn't 
> matter a
> bit. I started looking into scheduler tweaking when I decided to take 
> the
> path of least resistance and just install Linux instead. ZFS on Linux 
> on
> literally the same hardware, dealing with the same pools, handles this 
> same
> disk I/O without a hiccough.
> I've moved back to FreeBSD on this box now, and I'd like to resolve 
> this
> issue. I don't know if it's a matter of fixing something broken in 
> FreeBSD's
> scheduling or tuning ZFS somehow such that it's friendlier. (For what 
> it's
> worth, renice'd zfs processes don't make a bit of difference, just as 
> capping
> ARC didn't.)
> The box has FreeBSD 10.2, eight gigs of RAM, and I'm dealing with pools 
> 1TB
> or smaller. No deduplication. I'm not enough of a ZFS guru to have a 
> strong
> notion of what needs to change. I've not seen anything that seems
> particularly relevant in tuning guides. I have precious little 
> diagnostic
> data to share. That said, here's a quick idea of what FreeBSD is doing,
> captured last night, with my box doing precious little else beyond the
> transfer:
> last pid:  2631;  load averages:  9.44,  8.94,  7.28    up 0+00:29:03  
> 22:04:12
> 58 processes:  2 running, 56 sleeping
> CPU:  0.8% user,  0.0% nice, 95.8% system,  0.1% interrupt,  3.3% idle
> Mem: 136M Active, 14M Inact, 6915M Wired, 8784K Cache, 855M Free
> ARC: 6234M Total, 248M MFU, 5233M MRU, 626M Anon, 45M Header, 82M Other
> Swap: 8192M Total, 884K Used, 8191M Free
>  1334 root          1  52   15 42248K  3164K RUN     0   1:27   7.57% 
> zfs
>  1333 root          1  38   15 42248K  3232K pipewr  1   1:04   4.98% 
> zfs
> Not that this matters in grand scheme of things, but I'm hoping to get 
> a
> handle on what's happening here before frustration drives me back to 
> Linux.
> I'd be happy to gather diagnostics given some pointers on what would be
> useful. It seems unlikely that FreeBSD is this desperately inferior to 
> Linux
> in terms of the competency of its scheduler, but I'm not sure what to 
> tune to
> bring it up to the generally usable state I see on the same hardware 
> under
> Linux.
> Thanks in advance!

Try Starting here:

I have these settings set on my system running on an AMD FX(tm)-6300 
Six-Core Processor, with 16G of RAM, running 7 jails, and A Debian Linux 
Virtual Box VM, that runs Ubiquiti Unifi-Video software doing recording 
on 5 IP cameras. Running 4 1T SATA II drives in a raidz and a 2T SATA 
III drive with a zpool ontop of GELI for Bacula backups. ZFS Boot, with 
GPT partitions allocating a portion of each of the 1T drives as swap 
outside of the zpool. System works like a champ, its all standard 
desktop hardware, using on board SATA III ports All the drives are WD 
Green drives, so they are towards the low performance side.

# ZFS Options

    Dean E. Weimer

More information about the freebsd-questions mailing list