vm.kmem_size settings doesn't affect loader?

[Bartosz Stec]

Jeremy Chadwick pisze:
> On Thu, Sep 25, 2008 at 12:26:58PM +0200, Bartosz Stec wrote:
>   
>> Today I've experienced zfs-related kernel panic. Log says:
>>
>>    savecore: reboot after panic: kmem_malloc(131072): kmem_map too
>>    small: 327684096 total allocated
>>
>> Reported amount of memory (327684096) is wrong, because i made suggested  
>> tuning in my loader.conf:
>>
>>    vm.kmem_size="512M"
>>    vm.kmem_size_max="512M"
>>
>> Just to be sure:
>>
>>    # sysctl vm | grep kmem
>>    vm.kmem_size: 536870912
>>    vm.kmem_size_min: 0
>>    vm.kmem_size_max: 536870912
>>    vm.kmem_size_scale: 3
>>
>> Am I missing something?
>>     
>
> I believe this is normal.  The amount shown in "total allocated" will
> not match what you have vm.kmem_size or vm.kmem_size_max set to.
> Someone more familiar with the VM can explain why this is, but as I
> understand it, it's 100% normal.
>   
Thanks for explaining. This part of tuning guide confused me:

    "I was able to generate the following kernel panic in less than a
    minute by copying files from a linux server connected via gigabit
    crossover:
    Apr  8 06:46:08 nas savecore: reboot after panic: kmem_malloc(131072): kmem_map too small: 528273408 total allocated 
    As you can see, this is *with* vm.kmem_size="512M"!"

I've just expected similiar numbers in my case.
> Your options are:
>
> 1) Consider increasing it from 512M to something like 1.5GB; do not
> increase it past that on RELENG_7, as there isn't support for more than
> 2GB total.  For example, on a 1GB memory machine, I often recommend
> 768M.  On 2GB machines, 1536M.  You will need to run -CURRENT if you
> want more.
>
> 2) Tune ZFS aggressively.  Start by setting vfs.zfs.arc_min="16M"
> and vfs.zfs.arc_max="64M".
>
> If your machine has some small amount of memory (768MB, 1GB, etc.),
> then you probably shouldn't be using ZFS.
>
>   
Problem occured on i386 machine with 1GB of memory and 7.1-pre (3HDD, 
40GB, RAIDZ1). I know that i386 is highly unrecommended for ZFS, but 
it's just a home box for testing and learning purposes - I just want to 
know what I'm doing and what should I expect when I decide to put ZFS on 
server machines :) Currently, from posts on freebsd-fs, I conclude that 
even with a gigs of kmem and using AMD64, we still can experience panic 
from kmem_malloc. Manual tuning is hard for me because I'm not familiar 
with BSD kernel code nor kernel memory management. I'm just an end-user 
who love concepts of ZFS and wait for it to be (more) stable. Of course 
I've followed tuning guide carefully.
So, for now, I will add another 512MB of memory and increse vm.kmem_size 
and do more tests. Thanks once again for explaining and suggestions. 
Good luck with ZFS everyone! :)

-- 
Bartosz Stec