Troubleshooting kernel panic with zfs

Josh Gitlin jgitlin at
Fri Sep 21 15:17:46 UTC 2018

Thanks for the help Mike! In reviewing our settings I also found that we're setting vm.kmem_size to 1.5x physical RAM installed, based on a tuning guide someone found. I personally believe that might be misguided and may be related to the panic we saw, as this causes zfs-stats -a to show the kernel memory map being larger than system RAM:

Kernel Memory Map:				24.00	GiB
	Size:				21.72%	5.21	GiB
	Free:				78.28%	18.79	GiB

I'll try tuning these settings down and see if the problem happens again. Appreciate the support!

Josh Gitlin
Senior DevOps Engineer
(415) 690-1610 x155

Stay up to date and join the conversation in Relay <>.

> On Sep 21, 2018, at 9:04 AM, Mike Tancsa <mike at> wrote:
> On 9/20/2018 8:27 PM, Josh Gitlin wrote:
>> My hunch is that, given this was inside kmem_malloc, we were unable to allocate memory for a zio_write_compress call (the pool does have ZFS compression on) and hence this is a tuning issue and not a bug... but I am looking for confirmation and/or suggested changes/troubleshooting steps. The ZFS tuning configuration has been stable for years, to it may be a change in behavior or traffic... If this looks like it might be a bug, I will be able to get more information from a minidump if it reoccurs and can follow up on this thread.
> Given all the issues with memory pressure perhaps ARC is eating too
> much, I would try artificially limiting it so it does not eat up too
> much RAM.  We had a similar purposed server but with 32G RAM and I found
> one day the system was randomly shooting processes to reclaim RAM
> eg
> pid xxx (yyy), uid zzzz, was killed: out of swap space
> Perhaps
> sysctl -w vfs.zfs.arc_max=6000000000
> eg.
> 	---Mike
> -- 
> -------------------
> Mike Tancsa, tel +1 519 651 3400 x203
> Sentex Communications, mike at
> Providing Internet services since 1994
> Cambridge, Ontario Canada

More information about the freebsd-fs mailing list