FreeBSD 7.2-RC1 - ZFS related kernel panic "kmem_map too small"

[Adam McDougall]

Gary Jennejohn wrote:
> On Wed, 22 Apr 2009 00:34:00 +0200
> Kai Gallasch <gallasch at free.de> wrote:
>
> [snip a lot of stuff]
>   
>> In /usr/src/UPDATING I read:
>>
>> [..]
>>
>> 20090207:
>>         ZFS users on amd64 machines with 4GB or more of RAM should
>>         reevaluate their need for setting vm.kmem_size_max and
>>         vm.kmem_size manually.  In fact, after recent changes to the
>>         kernel, the default value of vm.kmem_size is larger than the
>>         suggested manual setting in most ZFS/FreeBSD tuning guides.
>>
>> So I understood this as "vm.kmem_size is set unnecessary large by
>> default. You should think about decreasing it to save some RAM"
>>
>> On my amd64 server the default values of kmem_size are
>>
>> vm.kmem_size_scale: 3
>> vm.kmem_size_max: 3865468109
>> vm.kmem_size_min: 0
>> vm.kmem_size: 1201446912
>>
>> Can someone give me a hint how to debug this problem further, or how to
>> find some reasonable values for setting vm.kmem_size_max and
>> vm.kmem_size with 16G of RAM?
>>
>>     
>
> Hmm, I wonder whether this applies to 7.2-RC1.  I don't know whether
> the kernel changes have been committed to 7.2 or whether they were
> already present when we started work on 7.2 because I haven't been
> paying much attention.
>
> On my 8-current amd64 machine with only 4GB of RAM I see larger values
> than you see with 16GB:
>
> sysctl vm.kmem_size_max
> vm.kmem_size_max: 4509713203
> sysctl vm.kmem_size
> vm.kmem_size: 1335824384
>
> ---
> Gary Jennejohn
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>
>   
It has been my experience that after the kmem maximums were raised to 
allow more than approx 1.6G kmem (a number of months ago), on some 
systems I still had to specifically raise the vm.kmem_size above the 
default otherwise I still got out of kmem panics far below the max.  I 
suspect there was pressure for kmem and it was unable to "raise" the 
limit fast enough, or maybe a fragmentation problem?  Additionally, 
depending on which host, I've found different limits to how high I can 
set the kmem settings on "recent" builds of 7 and 8 amd64, for example I 
have one 7.2 system with 4G ram and the kernel would panic if I booted 
with kmem=2G (1G works fine), but I have a 8.0 system with 2G ram and 
kmem=2G works fine.  Another 8.0 system has 6G ram but we could only 
boot successfully with kmem=3G, not 4G.