Significant memory leak in 9.3p10?

[J David]

On Mon, Mar 16, 2015 at 7:52 PM, J David <j.david.lists at gmail.com> wrote:
> On Mon, Mar 16, 2015 at 7:24 PM, Konstantin Belousov
> <kostikbel at gmail.com> wrote:
>> There are a lot of possibilities to create persistent anonymous shared
>> memory objects.  Not complete list is tmpfs mounts, swap-backed md disks,
>> sysv shared memory, possibly posix shared memory (I do not remember which
>> implementation is used in stable/9).
>
> If that's the explanation, how could it be
> detected/measured/investigated/resolved/prevented?
>
> Under ordinary circumstances, machines will go run like this for days/weeks:
>
> Mem: 549M Active, 3623M Inact, 567M Wired, 3484K Cache, 827M Buf, 3156M Free
> Swap: 1024M Total, 1024M Free
>
> Then, when this happens, it rapidly degrades from that to so bad that
> processes start getting killed for being out of swap space.

These FreeBSD machines running out of swap space and dying continues
to be a daily problem causing outages and unscheduled reboots.  Is
there really no way to even research what might be causing the
problem?

(Widening the cross-posting in the hopes of eliciting more help, so
the brief summary of the problem orginally posted to freebsd-stable is
that an unknown actor consumes all the user-space memory in the
system, including swap space, to the point where processes are killed
for being out of swap space, but if every process on the machine is
stopped, very little of the user-space memory in use is freed.
Original message with more details is here:
https://lists.freebsd.org/pipermail/freebsd-stable/2015-March/081986.html
.)

There are no tmpfs mounts or md disks, so it would have to be one of
the other causes.  How can FreeBSD's use of persistent, anonymous
shared memory objects be investigated, measured, or controlled so we
can get a handle on this issue?

Thanks!