10.1-BETA2 possible kernel memory leak in routing table

[Rumen Telbizov]

Hello everyone,

I think I got one step closer. At this point I am almost certain that the *leak
is related to PF reloading tables.*
Adding Gleb Smirnoff here since it might be something related to the
rewrite of PF in FreeBSD 10.

Here are the additional details.

The following dtrace script shows kernel memory leaks per probe:
#!/usr/sbin/dtrace -s
#pragma D option quiet

BEGIN {
    printf("%-20s %20s %20s %20s\n", "FUNCTION", "ALLOCATED", "FREE",
"TOTAL");
}

dtmalloc::$1:malloc {
    @malloc[probefunc] = sum(args[3]);
    @total[probefunc] = sum(args[3]);
}

dtmalloc::$1:free {
    @free[probefunc] = sum(args[3]);
    @total[probefunc] = sum(-args[3]);
}

tick-1s {
    printa("%-20.20s %20 at d %20 at d %20 at d\n", @malloc, @free, @total);
}

Running it like this:
# ./script2.d routetbl
FUNCTION                        ALLOCATED
FREE                TOTAL
routetbl                              512
512                    0
...
routetbl                            46592
46592                    0
routetbl                           808960
444416               364544
...
routetbl                           861184
496640               364544
routetbl                          1623552
894464               729088
...
routetbl                          1641984
912896               729088

The interval between the increase of 356KB is almost exactly 30 seconds.
This made me think about what I run on this machine every 30 seconds. I do
in fact have a service which every 30 seconds processes data, compiles PF
tables and reloads the PF tables like this: /sbin/pfctl -T load -f
/etc/firewall/pf.conf

I stopped this service and the amount of memory remained static! No more
leaks. I checked vmstat -m | grep routetbl and it had stopped growing. A
manual pfctl -f /etc/firewall/pf.conf causes a leak of 356KB of memory
every single time I run it.

Brian Somers and I are currently looking into the source of PF in latest
10-STABLE and trying to figure out what is going on. We were able to
replicate this problem on a 11-CURRENT (Sep 12th) machine as well. A simple
PF ruleset with 1 rule and 1 table. Every few reloads of the firewall
and vmstat
-m | grep routetbl shows increased memory usage.

For completeness - Alexander's dtrace script, with an aggregate twist,
shows this (and only this) after about a minute:
#!/usr/sbin/dtrace -s
fbt:kernel:rt_msg2:entry {
    @rt_msg2[stack()] = count();
}
fbt:kernel:rn_addroute:entry {
    @rn_addroute[stack()] = count();
}

Output:
              kernel`sysctl_rtsock+0x274
              kernel`sysctl_root+0x214
              kernel`userland_sysctl+0x1d8
              kernel`sys___sysctl+0x74
              kernel`amd64_syscall+0x334
              kernel`0xffffffff80900e0b
             2340

              kernel`sysctl_rtsock+0x64c
              kernel`sysctl_root+0x214
              kernel`userland_sysctl+0x1d8
              kernel`sys___sysctl+0x74
              kernel`amd64_syscall+0x334
              kernel`0xffffffff80900e0b
             4680

Getting pretty close ...

Regards,
-- 
Rumen Telbizov
Unix Systems Administrator <http://telbizov.com>