debugging mbuf allocation/dealocation

[Robert Watson]

On Sat, 27 Dec 2008, Nikola Knežević wrote:

> After running the Click in a configuration which generates 150k packets 
> (using m_dup) and discards them (m_freem), I get this in netstat -m output:
> 150258/1302/151560 mbufs in use (current/cache/total)
> 256/828/1084/4672 mbuf clusters in use (current/cache/total/max)
> 256/768 mbuf+clusters out of packet secondary zone in use (current/cache)
> 0/135/135/2336 4k (page size) jumbo clusters in use (current/cache/total/max)
> 0/0/0/1168 9k jumbo clusters in use (current/cache/total/max)
> 0/0/0/584 16k jumbo clusters in use (current/cache/total/max)
> 38076K/2521K/40598K bytes allocated to network (current/cache/total)
> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
> 0/0/0 requests for jumbo clusters denied (4k/9k/16k)
> 0/0/0 sfbufs in use (current/peak/max)
> 0 requests for sfbufs denied
> 0 requests for sfbufs delayed
> 0 requests for I/O initiated by sendfile
> 0 calls to protocol drain routines
>
> So, my question is: how to debug this? How can I track the usage of mbufs, 
> profile it, and such? I'm running Click as a kernel module.

Hi Nikola:

You can use the KTR(4) facility to trace memory allocations and deallocations, 
logging them to memory, disk, etc.  Unfortunately interpreting the data can be 
fairly tricky, as network leaks tend to happen over a long period of time, be 
stored in sockets, etc, but it's definitely possible and has been done. :-) 
Processing the results with a perl script goes a long way, as the 
allocated/freed pointers are included, etc.

Robert N M Watson
Computer Laboratory
University of Cambridge