Limiting mbuf memory.

[Jeff Roberson]

On Mon, 24 Nov 2008, Alfred Perlstein wrote:

> * Jeff Roberson <jroberson at jroberson.net> [081123 23:48] wrote:
>> I'm developing a patch for an alternate memory layout for mbuf clusters
>> that relies on contigmalloc.  Since this can fail, we'll still have to
>> retain the capability of allocating traditional clusters.  I'll report
>> details on that later.  I'm writing this email to address the issue of
>> resource accounting in mbufs.
>>
>> Presently we use a set of limits on individual zones or sizes of mbufs.
>> Standard mbufs, clusters, page size jumbos, 9k jumbos, and 16k jumbos.
>> Each is administered sperately.  I think this is getting a bit unwieldy.
>> In the future, we may have even more sizes.  This also introduces problems
>> because I will have two cluster zones do they each get their own limit?
>>
>> I would like to consolidate this into a single limit on the number of
>> pages in total allocated to networking.  With perhaps some fractional
>> reservation for standard mbufs and clusters to make sure they aren't
>> overwhelmed by the larger buffers.
>>
>> This would be implemented by overriding the uma zone page allocator for
>> each of the mbuf zones with one that counts pages for all.  Should we
>> reach the limit we'll block depending on the wait settings of the
>> requestor.  The limit and sleep will probably be protected by a global
>> lock which won't be an issue because trips to the back end allocator are
>> infrequent and protected by their own global lock anyhow.
>>
>> How do people feel about this?  To be clear this would eliminate:
>>
>> nmbclusters, nmbjumbop, nmbjumbo9, nmbjumbo16 and related config settings
>> and sysctls.  They would be replaced by something like 'maxmbufbytes'.
>> Presently we place no limit on small mbufs.  I could go either way on
>> this.  It could be added to the limit or not.
>
> This sounds good but please take into consideration the possibility
> of deadlock due to resource allocation to a single pool that can
> happen.
>
> It might make sense to keep the small and large mbuf limits separate
> or something like that.

This is what I meant in the third paragraph.

>
> Might also make sense to retain the limits but set them all to
> "unlimited" (withing the global limit) unless configured otherwise
> for various custom set ups.

I think this is a good idea.

>
> I don't feel too strongly about this, just some points to consider.

I appreciate the feedback.

Jeff

>
> -- 
> - Alfred Perlstein
>