mbuf autotuning effect

[hiren panchasara]

We are seeing an interesting thing on a mips board with 32MB ram.

We run out of mbuf very easily and looking at numbers it seems we are only
getting 6mb of maxmbufmem.

# sysctl -a | grep hw | grep mem
 hw.physmem: 33554432
 hw.usermem: 21774336
 hw.realmem: 33554432
 #
 # sysctl -a | grep maxmbuf
 kern.ipc.maxmbufmem: 6291456

I believe that number is very low for a board with 32mb of ram.

Looking at the code:

sys/kern/kern_mbuf.c : tunable_mbinit()

124         realmem = qmin((quad_t)physmem * PAGE_SIZE, vm_kmem_size);
125         maxmbufmem = realmem / 2;
126         TUNABLE_QUAD_FETCH("kern.ipc.maxmbufmem", &maxmbufmem);
127         if (maxmbufmem > realmem / 4 * 3)
128                 maxmbufmem = realmem / 4 * 3;

So, realmem plays important role in determining maxmbufmem.

physmem = 32mb
PAGE_SIZE = 4096

vm_kmem_size is calculated inside sys/kern/kern_malloc.c : kmeminit()

 705         vm_kmem_size = VM_KMEM_SIZE + nmbclusters * PAGE_SIZE;
 706         mem_size = cnt.v_page_count;
 707
 708 #if defined(VM_KMEM_SIZE_SCALE)
 709         vm_kmem_size_scale = VM_KMEM_SIZE_SCALE;
 710 #endif
 711         TUNABLE_INT_FETCH("vm.kmem_size_scale", &vm_kmem_size_scale);
 712         if (vm_kmem_size_scale > 0 &&
 713             (mem_size / vm_kmem_size_scale) > (vm_kmem_size /
PAGE_SIZE))
 714                 vm_kmem_size = (mem_size / vm_kmem_size_scale) *
PAGE_SIZE;

here,
VM_KMEM_SIZE = 12*1024*1024
nmbclusters = 0 (initially)
PAGE_SIZE = 4096
# sysctl -a | grep v_page_count
vm.stats.vm.v_page_count: 7035

and VM_KMEM_SIZE_SCALE = 3 for mips.

So, vm_kmem_size = 12mb.

Going back to tunable_mbinit(),
we get realmem = 12mb.
and masmbufmem = 6mb.


Wanted to see if I am following the code correctly and how autotuning
should work here.

cheers,
Hiren