mbuf leak with SMP and debug.mpsafenet=1

[Robert Watson]

On Tue, 19 Oct 2004, Andrew Gallatin wrote:

>  > I ran with this change in the netperf branch for quite a long time, but
>  > never managed to trigger sufficient races on the allocator to result in
>  > the counters getting off by more than a couple.  However, the reason I
>  > updated the patch and put it on the netperf page was that Bill Paul
>  > reported seeing fairly hefty stats errors on an SMP box at gig-e rates,
>  > and when he tried the patch it went away.  It would be useful if you could
>  > try the patch to make sure that we're looking at a real mbuf leak and not
>  > an mbuf stat leak.
> 
> Aha!
> 
> That seems to be it (a stats leak).  This is kind of a shame, since the
> last thing a P4 needs is more atomic ops :-(

Yeah -- I've been trying to avoid committing this patch since atomic
operations hurt the P4 quite a bit more than one would hope.  We already
do MPSAFE stats in UMA, so an interesting question might be whether these
stats are redundant to stats already gathered and we can use them instead. 
One of the theoretical advantages of mbuma is that mbufs become just
another case of existing slab allocated memory resources, so I would think
most of the interesting stats should be there. 

> I think there may have been a real leak in the past; at least I ran a
> box out of mbufs a week ago.  It only came back when I ifconfig'ed down
> my driver, freeing a bunch of mbufs.  But this was before green's recent
> mbuf leak fix, and in the middle of driver development.  So who knows.. 

If it was with if_em, the queueing bugs that were fixed recently may also
have helped.

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert at fledge.watson.org      Principal Research Scientist, McAfee Research