Fileserver panic - FreeBSD 8.1-stable and zfs

[Torfinn Ingolfsen]

On Thu, 02 Jun 2011 12:50:26 -0700
Jeremy Chadwick <freebsd at jdc.parodius.com> wrote:
> 
> This is a well-known thing with ZFS on FreeBSD.  Because you're running
> 8.1-STABLE, this makes figuring out all the tunables and so on a lot
> more difficult than if you were running 8.2-STABLE.

FWIW, the machine has been quite stable for me for a long time.

> Please provide:
> 
> 1) Contents of /boot/loader.conf

root at kg-f2# more /boot/loader.conf
zfs_load="YES"
vfs.root.mountfrom="zfs:zroot"
siis_load="YES"
amdtemp_load="YES"
# testing without MSI
hw.pci.enable_msix="0"
hw.pci.enable_msi="0"

> 2) Output from: sysctl hw.physmem hw.usermem hw.realmem (your hardware
>    page says 4GB, but I can't be bothered to sift through multi-pages
>    of wiki documents and links to find the answers)

root at kg-f2# sysctl hw.physmem hw.usermem hw.realmem
hw.physmem: 4141920256
hw.usermem: 3721527296
hw.realmem: 4966055936


> 3) Output from: sysctl vfs.zfs.zio.use_uma

root at kg-f2# sysctl vfs.zfs.zio.use_uma
vfs.zfs.zio.use_uma: 0


> The scrub itself was not ultimately responsible for this problem
> (meaning "the bug is not in scrub").  The problem is that your kernel
> effectively wanted more memory for ZFS operations than was available.

Understood. I didn't mean to imply it was; I just tried to provide data about activity on the server
that might have contributed to the failure.
FWIW, the scrub finished fine:
root at kg-f2# zpool status storage                                                                                       pool: storage
 state: ONLINE
 scrub: scrub completed after 307445734561825860h15m with 0 errors on Thu Jun  2 23:23:44 2011
config:

	NAME        STATE     READ WRITE CKSUM
	storage     ONLINE       0     0     0
	  raidz1    ONLINE       0     0     0
	    ad8     ONLINE       0     0     0
	    ad10    ONLINE       0     0     0
	    ad12    ONLINE       0     0     0
	    ad14    ONLINE       0     0     0
	    ada0    ONLINE       0     0     0

errors: No known data errors

> The "trick" is to tune /boot/loader.conf until you can gain stability.

Well, the server has been reasonably stable for me for about a year now
(I had to replace a failing hard drive, but I count that as "wear" not "instability").

> Again, because you're running 8.1-STABLE, the tuning parameters here
> will behave different than on 8.2-STABLE.  We can go over those in a
> follow-up thread.

I have no trouble with upgrading the server to 8.2-stable, if now is a good time to do it.
(I haven't watched closely for any zfs related problems on the mailing list lately.)

> I've gotten to the point where I literally cannot remember all of the
> different situations/conditions/tunings for each FreeBSD kernel build,
> release, date, type, etc., so I tend to focus on the most recent
> RELENG_8 build.  Then someone comes along with an older build.....
> Hehe.  :-)

I know what you mean. Keeping up with all this "stuff" is getting harder every year. :)
-- 
Torfinn