Limits on jumbo mbuf cluster allocation

[Garrett Wollman]

In article <513E3D75.7010803 at freebsd.org>, andre at freebsd.org writes:
>On 11.03.2013 17:05, Garrett Wollman wrote:
>> Well, I have two problems: one is running out of mbufs (caused, we
>> think, by ixgbe requiring 9k clusters when it doesn't actually need
>> them), and one is livelock.  Allowing potentially hundreds of clients
>> to queue 2 MB of requests before TCP pushes back on them helps to
>> sustain the livelock once it gets started, and of course those packets
>> will be of the 9k jumbo variety, which makes the first problem worse
>> as well.
>
>I think that TCP, or rather the send socket buffer, currently doesn't
>push back at all but simply accepts everything that gets thrown at it.

I'm talking about the receive side here -- the send side doesn't
really matter (and for NFS you probably want to have larger send
buffers than receive buffers anyway -- once you've invested all the
resources required to complete a request, you don't want it to fail at
the very last moment).  The normal TCP window mechanism provides all
the backpressure that's needed, if the buffers were not artificially
inflated.  It's very frustrating to configure a reasonable buffer size
for the application and then have the NFS code stomp all over the
configured values.

-GAWollman