mbuf autotuning effect

[Andre Oppermann]

On 07.09.2013 21:56, Ian Lepore wrote:
> On Sat, 2013-09-07 at 12:21 -0700, hiren panchasara wrote:
>> On Sep 6, 2013 8:26 PM, "Warner Losh" <imp at bsdimp.com> wrote:
>>>
>>>
>>> On Sep 6, 2013, at 7:11 PM, Adrian Chadd wrote:
>>>
>>>> Yeah, why is VM_KMEM_SIZE only 12mbyte for MIPS? That's a little
>> low
>> for a
>>>> platform that has a direct map that's slightly larger than 12mb :)
>>>>
>>>> Warner? Juli?
>>>
>>> All architectures have it at 12MB, except sparc64 where it is 16MB.
>> This
>> can be changed with the options VM_KMEM_SIZE=xxxxx in the config file.
>>
>> Right. Does that mean for any platform, if we do not have nmbclusters
>> pre-set in kmeminit() than we will always have pretty low value of
>> vm_kmem_size. And because of that, if maxmbufmem is not pre-set (via
>> loader.conf) inside tunable_mbinit() , we will have very low value for
>> maxmbufmem too.
>>
>> I hope (partially believe) that my understanding is not entirely
>> correct.
>> Because if its correct, we arw depending on loader.conf instead of
>> actually
>> auto tuning.
>>
> I think the part of this that strikes me as strange is calling 20% of
> physical memory used for network buffers a "very low value".  It seems
> outrageously high to me.   I'd be pissed if that much memory got wasted
> on network buffers on one of our $work platforms with so little memory.

This memory is NOT allocated to the mbuf system.  It is just the upper
*limit* how far it can go when the demand is there.

> So the fact that you think it's crazy-low and I think it's crazy-high
> may be a sign that it's auto-tuned to a reasonable compromise, and in
> both our cases the right fix would be to use the available knobs to tune
> things for our particular uses.

Yes.  The autotuning network memory *limit* is a compromise that allows
big beefy network servers work well right out of the box while not killing
small ones.  Lets say it covers > 90% of our actual users and use cases.

If you have special requirements then you may have to tune these values,
but if you known exactly what your system is going to do, then chances
are you're tuning various other parameters anyways.

-- 
Andre