Freebsd 8.0 kmem map too small

[Steve Polyack]

On 05/05/10 10:12, Ben Kelly wrote:
> On May 5, 2010, at 9:33 AM, Pawel Jakub Dawidek wrote:
>
>    
>> On Wed, May 05, 2010 at 10:46:31AM +0200, Giulio Ferro wrote:
>>      
>>> On 05.05.2010 09:52, Jeremy Chadwick wrote:
>>>
>>> Nope, it's happened again... Now I've tried to rise vm.kmem_size to 6G...
>>>
>>>
>>>        
>>>> Did you set both vm.kmem_size and vfs.zfs.arc_max, setting the latter to
>>>> something *less* than vm.kmem_size?
>>>>
>>>>
>>>>          
>>> Yes.
>>> After your suggestion, I set
>>> vfs.zfs.arc_max: 3758096384
>>> vm.kmem_size: 4G
>>>
>>> Now:
>>> vfs.zfs.arc_max: 3758096384
>>> vm.kmem_size: 6392119296
>>>        
>> Could you try to track down the commit that is causing your problems?
>> Could you try 8-STABLE kernel from before r206815?
>>      
>
> Are others generally able to run ARC values so close to kmem size?  My experience has been that you really need the ARC to be much smaller than the kmem limit (like 25% or less) due to fragmentation of kmem_map.
>
>    
On a system here with 8GB of RAM and a very large zpool consisting of 
multiple zdevs, little tuning was needed.  The system is running 
8-STABLE(amd64) as of a month or two ago.  The only things I have set in 
/boot/loader.conf are:
vm.kmem_size="12G"
vfs.zfs.arc_max="4G"

Setting kmem_size to 12G came from a combination of recommendations I 
saw in various mailing list posts (you can probably dig them up).  
Setting it to the physical memory size was the initial recommendation, 
while others recommended 1.5x physical memory size to help prevent 
fragmentation/wasted space in kmem.

Regardless, this has served us quite well for the ~6 months the system 
has been in use.  It has never crashed, even under intensive 
multi-threaded benchmarking.
-Steve