Limits on jumbo mbuf cluster allocation

[Garrett Wollman]

<<On Fri, 8 Mar 2013 19:47:13 -0500 (EST), Rick Macklem <rmacklem at uoguelph.ca> said:

> If reducing the size to 4K doesn't fix the problem, you might want to
> consider shrinking the tunable vfs.nfsd.tcphighwater and suffering
> the increased CPU overhead (and some increased mutex contention) of
> calling nfsrv_trimcache() more frequently.

Can't do that -- the system becomes intolerably slow when it gets into
that state, and seems to get stuck that way, such that the only way to
restore performance is to increase the size of the "cache".
(Essentially all of the nfsd service threads end up spinning most of
the time, load average goes to N, and goodput goes to nearly nil.)  It
does seem like a lot of effort for an extreme edge case that, in
practical terms, never happens.

> (I'm assuming that you are using drc2.patch + drc3.patch.

I believe that's what I have.  If my kernel coding skills were less
rusty, I'd fix it to have a separate cache-trimming thread.

One other weird thing that I've noticed is that netstat(1) reports the
send and receive queues on NFS connections as being far higher than I
have the limits configured.  Does NFS do something to override this?

-GAWollman