Re: git: eb93b99d6986 - main - in_pcb: delay crfree() down into UMA dtor

On Mon, 27 Dec 2021 16:46:54 -0800
Gleb Smirnoff <glebius@freebsd.org> wrote:

> On Fri, Dec 17, 2021 at 10:17:55PM -0800, Gleb Smirnoff wrote:
> T> T> K> The jails created by that test never go away. It’s as if 
> T> T> K> `crfree(inp->inp_cred);` doesn’t actually get called. And
> T> T> K> indeed, it looks like inpcb_dtor() does not get called at
> T> T> K> all.  
> T> T> 
> T> T> Yes, I faced this problem today, too. :(
> T> T> 
> T> T> My radical opinion is that per-VNET pcb zones should just be
> T> T> eliminated. The only thing they serve is imposing maxsockets
> T> T> limit separately for each VNET. But we already have the
> T> T> maxsocket limit on the socket zone, which is _global_!
> T> T> 
> T> T> Anybody to explain me the sense of the per-VNET per-pcb zone
> T> T> limit set to the same maxsockets value? You can't create a pcb
> T> T> without a socket, which is guaranteed by the in_pcballoc()
> T> T> prototype. Of course I understand that pcbs may outlive the
> T> T> socket. But those pcbs that outlive a socket, are eventually
> T> T> garbage collected as their lifetime is finite. Anyway jail/VNET
> T> T> was never declared as a resource management framework anyway!
> T> T> 
> T> T> So, for this particular problem I would suggest just eliminate
> T> T> per-VNET pcb zones, but in general the fact that idle SMR zone
> T> T> may never purge its cache sucks and needs improvement.  
> T> 
> T> I have created a patch that would mitigate that problem. Once the
> T> zones are global, the jails will eventually die if there is some
> T> pcb zone traffic.
> T> 
> T> https://reviews.freebsd.org/D33542  
> 
> Despite I still believe that PCB zones belong to global state rather
> than to a VNET, the patch doesn't help to mitigate massive memory
> leaks with vnet jails on a machine that is dedicated solely to run a
> test suite. If machine does nothing except a test suite, there is
> almost zero pcb traffic. If there is no pcb zone traffic, the SMR
> caches stay, and thus destroyed jails will also stay. Our vnet jail
> "weights" a lot! Even with the global PCB zone patch applied, each
> vnet jail creates 33 UMA zones!
> 
> I think we need a KPI to purge the SMR caches, and we also need to put
> vnet jails on a diet. These are two independent problems, of course.

+1 for nuking all per-vnet PCB zones and the alike!  At the time I
V_irtualized them during the early stages of VNET implementaion, the
focus was on correctnes and tracking of inter-vnet resource leaks. Once
that step was reasonably completed (circa 15 years ago!), per-VNET
zones became a pure waste of memory, amplified with per-CPU local free
pools for each zone, not to mention the PITA with VNET cleanups...

If memory still serves me well, a few folks asserted that per-VNET zones
could be useful for hypothetical VNET snapshots / live migration to
another machine, a project I heard about on several occasions but have
never seen it...

So, by all means go ahead and devirtualize them all...

Cheers,

Marko