9k jumbo clusters

[John-Mark Gurney]

Ryan Moeller wrote this message on Fri, Jul 27, 2018 at 12:45 -0700:
> There is a long-standing issue with 9k mbuf jumbo clusters in FreeBSD.
> For example:
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183381
> https://lists.freebsd.org/pipermail/freebsd-net/2013-March/034890.html
> 
> This comment suggests the 16k pool does not have the fragmentation problem:
> https://reviews.freebsd.org/D11560#239462
> I???m curious whether that has been confirmed.
> 
> Is anyone working on the pathological case with 9k jumbo clusters in the
> physical memory allocator?  There was an interesting discussion started a
> few years ago but I???m not sure what ever came of it:
> http://docs.freebsd.org/cgi/mid.cgi?21225.20047.947384.390241
> 
> I have seen some work in the direction of avoiding larger than page size
> jumbo clusters in 12-CURRENT.  Many existing drivers avoid the 9k cluster
> size already.  The code for larger cluster sizes in iflib is #ifdef'd out
> so it maxes out at the page size jumbo clusters until "CONTIGMALLOC_WORKS"
> (apparently it doesn't).
> 
> With all the changes due to iflib, is there any chance some of this will
> get MFC'd to address the serious problem that remains in 11-STABLE?
> 
> Otherwise, would it be feasible to disable the use of the 9k cluster pool
> in at least some of the popular NIC drivers as a solution for the stable
> branches?
> 
> Finally, I have studied some of the driver code in 11-STABLE and posted the
> gist of my notes in relation to this problem.  If anyone spots a mistake or
> has something else to contribute, comments on the gist would be greatly
> appreciated!
> https://gist.github.com/freqlabs/eba9b755f17a223260246becfbb150a1

Drivers need to be fixed to use 4k pages instead of cluster.  I really hope
no one is using a card that can't do 4k pages, or if they are, then they
should get a real card that can do scatter/gather on 4k pages for jumbo
frames..

-- 
  John-Mark Gurney				Voice: +1 415 225 5579

     "All that I will do, has been done, All that I have, has not."