Problems with ZFS file servers

[Matthew Seaman]

On 25/09/2015 20:42, Bob Healey wrote:
> I've got another machine acting up, this machine is a Sun X2250,
> originally was running Solaris until this summer when the owner dropped
> the support contract.  A zpool export, reinstall to FreeBSD 10.1, and
> zpool import and it was back in business.  Most of the requested info
> can be found at http://origami.phys.rpi.edu/~healer/lepton.  I am
> working on getting cacti installed.  I am currently trying to rsync a
> workstation to this pool so I can reload the OS.  If I use
> --bwlimit=10240 or lower, I have no issues, but if I don't rsync freezes
> on me on the client side.

I've found, through bitter experience, that you need to apply some
tunings to ZFS machines, and quite possibly some kernel patches too.
When you're pumping wads of data into a ZFS machine at high speed, it is
all too easy to get it to lock up.

First up, the default setting where ZFS grabs all but 1GB of available
RAM for use by the ARC is nuts.  You need to chop that down and give the
rest of the OS a fair share of RAM to play with by setting
vfs.zfs.arc_max in /boot/loader.conf.  What you set it to depends on the
application mix on your server, but somewhere around 50% of available
RAM seems reasonable to me.  Reboot to enable that, obviously.

If you're dumping a lot of data (especially if you're writing much more
than will fit into system RAM) onto your ZFS box, and don't intend to be
using that data actively any time soon, then it's a good idea to set
properties to disable the primarycache and secondarycache on the ZFS
you're writing to.  Or set them to metadata only.  Otherwise you'll just
blow out your ARC caches which won't help system performance.  You can
turn caching on or off for specific ZFSes at any time.

If that doesn't do enough to make your machine work reliably, then
investigate using the patches from PR 187594:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187594

There are several versions of the patch but you should be able to find
one that applies to the version of the OS you're using.

	Cheers,

	Matthew


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 957 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-questions/attachments/20150925/1d7448b9/attachment.bin>