threads/160708: Bypass process stack quota :)

[John Baldwin]

On Monday, September 19, 2011 9:47:57 am Michael Pounov wrote:
> On Mon, 19 Sep 2011 08:43:27 -0400
> John Baldwin <jhb at freebsd.org> wrote:
> 
> > On Saturday, September 17, 2011 7:50:08 am Michael Pounov wrote:
> > > The following reply was made to PR threads/160708; it has been noted by 
> > GNATS.
> > > 
> > > From: Michael Pounov <misho at elwix.org>
> > > To: freebsd-gnats-submit at freebsd.org
> > > Cc:  
> > > Subject: Re: threads/160708: Bypass process stack quota :)
> > > Date: Sat, 17 Sep 2011 14:26:11 +0300
> > > 
> > >  Hmm, you no so right Peter. 
> > >  
> > >  	Yes I can move esp pointer in any other address, but please 
> > >  start program and see address of allocated memory for every thread. 
> > >  All this allocations is made in upper memory called stack. 
> > >  	
> > >  	Try same alloca() in main program thread and you see how 
> > >  system terminate program if you going over stack limit.
> > 
> > It's not very practical to apply this limit to multithreaded apps.  Would you 
> > want it to be a global limit (i.e. all stacks summed together must be <= 
> > limit) or a per-thread limit (i.e. each thread's stack must be <= limit).  
> > Also, given that RLIMIT_DATA is now obsolete (since malloc() defaults to using 
> > MAP_ANON with mmap() rather than sbrk()), using RLIMIT_AS is probably the 
> > right thing if you are trying to prevent local DOS.
> > 
> > -- 
> > John Baldwin
> 
> My opinion from security viewpoint is all stacks summed together must be <= limit.
> Also reasonable solution is each thread's stack must be <= limit.

Yes, both are reasonable, which is part of why it is hard to apply the limit
in a generic sense.  Someone will get bitten by assuming it works one way when
it actually works another.

> Problem is not from own software, problem appears from buggy software of customer installed on server. I am not happy with that, when customer 
able get all resources with user account.
> This should be avoided when posible.

And RLIMIT_STACK is not a good tool for this.  RLIMIT_AS is a much better tool
this limits the total virtual memory usage including stack, text, data, and
the heap while allowing individual programs more freedom with what mix they
use.

That is, if you just want to limit processes to 8GB of memory, do you really
care if it uses 4GB of stack and 2GB of heap vs 512MB of stack and 5.5GB of
heap?  I don't think an admin should care about that, what they really want to
limit is the amount of memory used so they can avoid swapping, etc.

-- 
John Baldwin