ZFS gets stuck
Adam McDougall
mcdouga9 at egr.msu.edu
Sun Dec 7 19:24:42 PST 2008
On Sun, Dec 07, 2008 at 11:21:13AM +0100, Stefan Bethke wrote:
For the past week, I've been stress testing two new boxes by running
make -j4 universe. /usr/src is on ufs, /usr/obj on zfs, backed by a
single disk pool. Every so often (about once every one or two days),
processes start getting wedged on accessing the zfs file systems.
FreeBSD lokschuppen.lassitu.de 8.0-CURRENT FreeBSD 8.0-CURRENT #1: Wed
Dec 3 07:05:03 UTC 2008 root at lokschuppen.lassitu.de:/usr/obj/usr/
src/sys/EISENBOOT amd64
So far, I've had this in loader.conf:
vfs.zfs.arc_max="512M"
vfs.zfs.prefetch_disable="1"
I'm now adding
vfs.zfs.zil_disable="1"
to see if that makes a difference.
Is there anything in particular people would want me to check out?
Kernel is GENERIC minus a number of devices, and without INVARIANTS
and WITNESS.
I have one system running a recent -current with 2G of ram that would
tend to get stuck almost nightly during a nightly rsync, and have been
trying to tune it to get rid of the problems. I've made it about 6 days
so far using just:
vm.kmem_size=2G
vm.kmem_size_max=2G
vfs.zfs.arc_max=512M
With these settings, I had two rsyncs running constantly for probably 12
hours on the first day or two and still once nightly, so far so good. I
tried leaving the kmem settings out and while vm.kmem_size_max appeared to
auto tune to approx 4.5G, I still had a out of kmem panic (a low number
around 600 or 900M, I don't remember) until I boosted both kmem settings to
2G. I previously had ~1.6G for the kmem settings from when that was the
highest they could be set, and maybe that wasn't high enough with recent
-current because it had hangs every few days as opposed to every few months
with zfs "6" (same zfs as in -stable). I might just be having a string of
good luck, and the data that rsync transfers might have played a role (it
changes a varying amount each day). If it looks fairly stable then I will
apply the settings to a different, busier server with ZFS.
More information about the freebsd-current
mailing list