limit process memory usage

[Gerrit Kühn]

Hello all,

I have an application that sometimes develops some kind of memory leak
or similar and eats up all RAM within a few minutes until the system is
running out of memory and swap so the kernel starts randomly killing other
processes and finally the crashes.
Is there a way to limit the memory available to an (or any) application so
that something like this doesn't tear down the whole system every time it
happens but just kills the culprit? I found the rctl tool, but I couldn't
make out how to use it for this purpose so far.


cu
  Gerrit