ZFS vs OSX Time Machine
doconnor at gsoft.com.au
Fri Apr 29 00:13:57 UTC 2011
On 29/04/2011, at 5:26, Jeremy Chadwick wrote:
>> I have the following ZFS related tunables
> Are the last two actually *working* in /boot/loader.conf? Can you
> verify by looking at them via sysctl? AFAIK they shouldn't work, since
> they lack double-quotes around the values. Parsing errors are supposed
> to throw you back to the loader prompt. See loader.conf(5) for the
Yep, they're working :)
> I'm also not sure why you're setting cache_flush_disable at all.
I think I was wondering if it would help the abysmal write performance of these disks..
>> Any help appreciated, thanks :)
> Others seem to be battling stating that "NFS doesn't work for TM", but
> that isn't what you're complaining about. You're complaining that
> FreeBSD with ZFS + NFS performs extremely poorly when trying to do
> backups from an OS X client using TM (writing to the NFS mount).
Yes, and also TM is over AFP not NFS (I forgot to mention that..)
> I have absolutely no experience with TM or OS X, so if it's actually a
> client-level problem (which I'm doubting) I can't help you there.
> Just sort of a ramble here at different things...
> It would be useful to provide ZFS ARC sysctl data from the FreeBSD
> system where you're seeing performance issues. "sysctl -a
> kstat.zfs.misc.arcstats" should suffice.
> You should also try executing "zpool iostat -v 1" during the TM backup
> to see if there's a particular device which is behaving poorly. There
> have been reports of ZFS pools behaving poorly when a single device
> within the pool has slow I/O (e.g. 5 hard disks, one of which has
> internal issues, resulting in the entire pool performing horribly). You
> should let this run for probably 60-120 seconds to get an idea. Given
> your parameters above (assuming vfs.zfs.txg.timeout IS in fact 5!), you
> should see "bursts" of writes every 5 seconds.
> I know that there are some things on ZFS that perform badly overall.
> Anything that involves excessive/large numbers of files (not file sizes,
> but actual files themselves) seems to perform not-so-great with ZFS.
> For example, Maildir on ZFS = piss-poor performance. There are ways to
> work around this issue (if I remember correctly, by adding a dedicated
> "log" device to your ZFS pool, but be aware your log devices need to
> be reliable (if you have a single log device and it fails the entire
> pool is damaged, if I remember right)), but I don't consider it
> feasible. So if TM is creating tons of files on the NFS mount (backed
> by ZFS), then I imagine the performance isn't so great.
Hmm, the sparse disk image does have ~80000 files in a single directory..
> Could you please provide the following sysctl values? Thanks.
> If the FreeBSD machine has a wireless card in it, if at all possible
> could you try ruling that out by hooking up wired Ethernet instead?
> It's probably not the cause, but worth trying anyway. If you have a
> home router or something doing 802.11, don't bother with this idea.
The FreeBSD box is wired, although it's using an re card as the em card died(!!).
The OSX box is connected via an Airport Express (11n).
> Next, you COULD try using Samba/CIFS on the FreeBSD box to see if you
> can narrow the issue down to bad NFS performance. Please see this post
> of mine about tuning Samba on FreeBSD (backed by ZFS) to get extremely
> good performance. Many people responded and said their performance
> drastically improved (you can see the thread yourself). The trick is
> AIO. You can ignore the part about setting vm.kmem_size in loader.conf;
> that advice is now old/deprecated (does not pertain to you given the
> date of your kernel), and vfs.zfs.txg.write_limit_override is something
> you shouldn't mess with unless absolutely needed to leave it default:
OK. I don't think TM can use CIFS, I will try ISCSI as someone else suggested, perhaps it will help.
> Finally, when was the last time this FreeBSD machine was rebooted? Some
> people have seen horrible performance that goes away after a reboot.
> There's some speculation that memory fragmentation has something to do
> with it. I simply don't know. I'm not telling you to reboot the box
> (please don't; it would be more useful if it could be kept up in case
> folks want to do analysis of it).
I think performance does improve after a reboot :(
top looks like..
last pid: 16112; load averages: 0.24, 0.22, 0.23 up 8+16:11:50 09:43:19
653 processes: 1 running, 652 sleeping
CPU: 3.6% user, 0.0% nice, 3.4% system, 0.6% interrupt, 92.5% idle
Mem: 1401M Active, 578M Inact, 4143M Wired, 4904K Cache, 16M Buf, 1658M Free
Swap: 4096M Total, 160M Used, 3936M Free, 3% Inuse
although free does go down very low (~250MB) at times.
Daniel O'Connor software and network engineer
for Genesis Software - http://www.gsoft.com.au
"The nice thing about standards is that there
are so many of them to choose from."
-- Andrew Tanenbaum
GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C
More information about the freebsd-stable