Kernel memory leak in 8.2-PRERELEASE?

[Boris Kochergin]

On 04/02/11 11:41, Boris Kochergin wrote:
> On 04/02/11 11:33, Kostik Belousov wrote:
>> On Sat, Apr 02, 2011 at 10:17:27AM -0400, Boris Kochergin wrote:
>>> Ahoy. This morning, I awoke to the following on one of my servers:
>>>
>>> pid 59630 (httpd), uid 80, was killed: out of swap space
>>> pid 59341 (find), uid 0, was killed: out of swap space
>>> pid 23134 (irssi), uid 1001, was killed: out of swap space
>>> pid 49332 (sshd), uid 1001, was killed: out of swap space
>>> pid 69074 (httpd), uid 0, was killed: out of swap space
>>> pid 11879 (eggdrop-1.6.19), uid 1001, was killed: out of swap space
>>> ...
>>>
>>> And so on.
>>>
>>> The machine is:
>>>
>>> FreeBSD exodus.poly.edu 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE #2: Thu
>>> Dec  2 11:39:21 EST 2010
>>> spawk at exodus.poly.edu:/usr/obj/usr/src/sys/EXODUS  amd64
>>>
>>> 10:13AM  up 120 days, 20:06, 2 users, load averages: 0.00, 0.01, 0.00
>>>
>>> The memory line from top intrigued me:
>>>
>>> Mem: 16M Active, 48M Inact, 6996M Wired, 229M Cache, 828M Buf, 605M 
>>> Free
>>>
>>> The machine has 8 gigs of memory, and I don't know what all that wired
>>> memory is being used for. There is a large-ish (6 x 1.5-TB) ZFS RAID-Z2
>>> on it which has had a disk in the UNAVAIL state for a few months:
>>>
>>> # zpool status
>>>    pool: home
>>>   state: DEGRADED
>>> status: One or more devices could not be used because the label is
>>> missing or
>>>          invalid.  Sufficient replicas exist for the pool to continue
>>>          functioning in a degraded state.
>>> action: Replace the device using 'zpool replace'.
>>>     see: http://www.sun.com/msg/ZFS-8000-4J
>>>   scrub: none requested
>>> config:
>>>
>>>          NAME        STATE     READ WRITE CKSUM
>>>          home        DEGRADED     0     0     0
>>>            raidz2    DEGRADED     0     0     0
>>>              ada0    ONLINE       0     0     0
>>>              ada1    ONLINE       0     0     0
>>>              ada2    ONLINE       0     0     0
>>>              ada3    ONLINE       0     0     0
>>>              ada4    ONLINE       0     0     0
>>>              ada5    UNAVAIL      0    85    11  experienced I/O 
>>> failures
>>>
>>> errors: No known data errors
>>>
>>> "vmstat -m" and "vmstat -z" output:
>>>
>>> http://acm.poly.edu/~spawk/vmstat-m.txt
>>> http://acm.poly.edu/~spawk/vmstat-z.txt
>>>
>>> Anyone have a clue? I know it's just going to happen again if I reboot
>>> the machine. It is still up in case there are diagnostics for me to 
>>> run.
>> Try r218795. Most likely, your issue is not leak.
>
> Thanks. Will update to today's 8-STABLE and report back.
>
> -Boris

The problem persists, I'm afraid, and seems to have crept up a lot more 
quickly than before:

# uname -a
FreeBSD exodus.poly.edu 8.2-STABLE FreeBSD 8.2-STABLE #3: Sat Apr  2 
11:48:43 EDT 2011     spawk at exodus.poly.edu:/usr/obj/usr/src/sys/EXODUS  
amd64

Mem: 314M Active, 955M Inact, 6356M Wired, 267M Cache, 828M Buf, 18M Free

Any ideas for a diagnostic recourse?

-Boris