9k jumbo clusters

[Ryan Moeller]

Hello,

There is a long-standing issue with 9k mbuf jumbo clusters in FreeBSD.
For example:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183381
https://lists.freebsd.org/pipermail/freebsd-net/2013-March/034890.html

This comment suggests the 16k pool does not have the fragmentation problem:
https://reviews.freebsd.org/D11560#239462
I’m curious whether that has been confirmed.

Is anyone working on the pathological case with 9k jumbo clusters in the
physical memory allocator?  There was an interesting discussion started a
few years ago but I’m not sure what ever came of it:
http://docs.freebsd.org/cgi/mid.cgi?21225.20047.947384.390241

I have seen some work in the direction of avoiding larger than page size
jumbo clusters in 12-CURRENT.  Many existing drivers avoid the 9k cluster
size already.  The code for larger cluster sizes in iflib is #ifdef'd out
so it maxes out at the page size jumbo clusters until "CONTIGMALLOC_WORKS"
(apparently it doesn't).

With all the changes due to iflib, is there any chance some of this will
get MFC'd to address the serious problem that remains in 11-STABLE?

Otherwise, would it be feasible to disable the use of the 9k cluster pool
in at least some of the popular NIC drivers as a solution for the stable
branches?

Finally, I have studied some of the driver code in 11-STABLE and posted the
gist of my notes in relation to this problem.  If anyone spots a mistake or
has something else to contribute, comments on the gist would be greatly
appreciated!
https://gist.github.com/freqlabs/eba9b755f17a223260246becfbb150a1

Thanks in advance!

Ryan