mbuf autotuning effect

[hiren panchasara]

On Sat, Sep 7, 2013 at 12:56 PM, Ian Lepore <ian at freebsd.org> wrote:

> On Sat, 2013-09-07 at 12:21 -0700, hiren panchasara wrote:
> > On Sep 6, 2013 8:26 PM, "Warner Losh" <imp at bsdimp.com> wrote:
> > >
> > >
> > > On Sep 6, 2013, at 7:11 PM, Adrian Chadd wrote:
> > >
> > > > Yeah, why is VM_KMEM_SIZE only 12mbyte for MIPS? That's a little
> > low
> > for a
> > > > platform that has a direct map that's slightly larger than 12mb :)
> > > >
> > > > Warner? Juli?
> > >
> > > All architectures have it at 12MB, except sparc64 where it is 16MB.
> > This
> > can be changed with the options VM_KMEM_SIZE=xxxxx in the config file.
> >
> > Right. Does that mean for any platform, if we do not have nmbclusters
> > pre-set in kmeminit() than we will always have pretty low value of
> > vm_kmem_size. And because of that, if maxmbufmem is not pre-set (via
> > loader.conf) inside tunable_mbinit() , we will have very low value for
> > maxmbufmem too.
> >
> > I hope (partially believe) that my understanding is not entirely
> > correct.
> > Because if its correct, we arw depending on loader.conf instead of
> > actually
> > auto tuning.
> >
> I think the part of this that strikes me as strange is calling 20% of
> physical memory used for network buffers a "very low value".  It seems
> outrageously high to me.   I'd be pissed if that much memory got wasted
> on network buffers on one of our $work platforms with so little memory.
>

Interesting. So here how it looks on my laptop running amd64 GENERIC looks
like: (without any special loader.conf settings)

flymockour-l7% uname -a
FreeBSD flymockour-l7.corp.yahoo.com 10.0-CURRENT FreeBSD 10.0-CURRENT #1
r253512M: Sat Jul 20 23:00:51 PDT 2013
hirenp at flymockour-l7.corp.yahoo.com:/usr/obj/usr/home/hirenp/head/sys/GENERIC
amd64

flymockour-l7% sysctl -a | grep hw| grep mem
hw.physmem: 8496877568
hw.usermem: 3538432000
hw.realmem: 9093251072

flymockour-l7% sysctl kern.ipc.maxmbufmem
kern.ipc.maxmbufmem: 4132540416
flymockour-l7% sysctl -a | grep vm.kmem_
vm.kmem_size: 8265080832
vm.kmem_size_min: 0
vm.kmem_size_max: 329853485875
vm.kmem_size_scale: 1
vm.kmem_map_size: 1380515840
vm.kmem_map_free: 5796265984

VM_KMEM_SIZE_SCALE is 1 for amd64 while 3 for mips. Which might be one
reason.


> So the fact that you think it's crazy-low and I think it's crazy-high
> may be a sign that it's auto-tuned to a reasonable compromise, and in
> both our cases the right fix would be to use the available knobs to tune
> things for our particular uses.
>

I am pretty ignorant on what the value _should_ be. I will try to find out
more.

cheers,
Hiren