Limits on jumbo mbuf cluster allocation

[Andre Oppermann]

On 11.03.2013 17:05, Garrett Wollman wrote:
> In article <513DB550.5010004 at freebsd.org>, andre at freebsd.org writes:
>
>> Garrett's problem is receive side specific and NFS can't do much about it.
>> Unless, of course, NFS is holding on to received mbufs for a longer time.
>
> Well, I have two problems: one is running out of mbufs (caused, we
> think, by ixgbe requiring 9k clusters when it doesn't actually need
> them), and one is livelock.  Allowing potentially hundreds of clients
> to queue 2 MB of requests before TCP pushes back on them helps to
> sustain the livelock once it gets started, and of course those packets
> will be of the 9k jumbo variety, which makes the first problem worse
> as well.

I think that TCP, or rather the send socket buffer, currently doesn't
push back at all but simply accepts everything that gets thrown at it.
This obviously is a problem and the NFS server seems to depend somewhat
on that by requiring atomicity on a RPC send.  I have to trace the mbuf
path through NFS to the socket to be sure.  The code is slightly opaque
though.

-- 
Andre