vm.kmem_size_max and vm.kmem_size capped at 329853485875 (~307GB)

[Alan Cox]

On 8/20/2012 8:26 PM, Gezeala M. Bacuño II wrote:
> On Mon, Aug 20, 2012 at 9:07 AM, Gezeala M. Bacuño II <gezeala at gmail.com> wrote:
>> On Mon, Aug 20, 2012 at 8:22 AM, Alan Cox <alc at rice.edu> wrote:
>>> On 08/18/2012 19:57, Gezeala M. Bacuño II wrote:
>>>> On Sat, Aug 18, 2012 at 12:14 PM, Alan Cox<alc at rice.edu>  wrote:
>>>>> On 08/17/2012 17:08, Gezeala M. Bacuño II wrote:
>>>>>> On Fri, Aug 17, 2012 at 1:58 PM, Alan Cox<alc at rice.edu>   wrote:
>>>>>>> vm.kmem_size controls the maximum size of the kernel's heap, i.e., the
>>>>>>> region where the kernel's slab and malloc()-like memory allocators
>>>>>>> obtain
>>>>>>> their memory.  While this heap may occupy the largest portion of the
>>>>>>> kernel's virtual address space, it cannot occupy the entirety of the
>>>>>>> address
>>>>>>> space.  There are other things that must be given space within the
>>>>>>> kernel's
>>>>>>> address space, for example, the file system buffer map.
>>>>>>>
>>>>>>> ZFS does not, however, use the regular file system buffer cache. The
>>>>>>> ARC
>>>>>>> takes its place, and the ARC abuses the kernel's heap like nothing
>>>>>>> else.
>>>>>>> So, if you are running a machine that only makes trivial use of a
>>>>>>> non-ZFS
>>>>>>> file system, like you boot from UFS, but store all of your data in ZFS,
>>>>>>> then
>>>>>>> you can dramatically reduce the size of the buffer map via boot loader
>>>>>>> tuneables and proportionately increase vm.kmem_size.
>>>>>>>
>>>>>>> Any further increases in the kernel virtual address space size will,
>>>>>>> however, require code changes.  Small changes, but changes nonetheless.
>>>>>>>
>>>>>>> Alan
>>>>>>>
>> <<snip>>
>>>>> Your objective should be to reduce the value of "sysctl vfs.maxbufspace".
>>>>> You can do this by setting the loader.conf tuneable "kern.maxbcache" to
>>>>> the
>>>>> desired value.
>>>>>
>>>>> What does your machine currently report for "sysctl vfs.maxbufspace"?
>>>>>
>>>> Here you go:
>>>> vfs.maxbufspace: 54967025664
>>>> kern.maxbcache: 0
>>>
>>> Try setting kern.maxbcache to two billion and adding 50 billion to the
>>> setting of vm.kmem_size{,_max}.
>>>
> 2 : 50 ==>> is this the ratio for further tuning
> kern.maxbcache:vm.kmem_size? Is kern.maxbcache also in bytes?
>

No, this is not a ratio.  Yes, kern.maxbcache is in bytes. Basically, 
for every byte that you subtract from vfs.maxbufspace, through setting 
kern.maxbcache, you can add a byte to vm.kmem_size{,_max}.

Alan