svn - but smaller?

[Stephen Montgomery-Smith]

On 02/24/2013 05:43 PM, Ian Lepore wrote:
> On Sun, 2013-02-24 at 13:24 -0800, Jeremy Chadwick wrote:
>> On Sun, Feb 24, 2013 at 10:19:57AM -0700, Ian Lepore wrote:
>>> On Sat, 2013-02-23 at 22:31 -0800, Jeremy Chadwick wrote:
>>>>
>>>> Also, John, please consider using malloc(3) instead of heap-allocated
>>>> buffers like file_buffer[6][] (196608 bytes) and command[] (32769
>>>> bytes).  I'm referring to this: 
>>>
>>> Why?  I absolutely do not understand why people are always objecting to
>>> large stack-allocated arrays in userland code (sometimes with the
>>> definition of "large" being as small as 2k for some folks).
>>
>> This is incredibly off-topic, but I'll bite.
>>
>> I should not have said "heap-allocated", I was actually referring to
>> statically-allocated.
>>
>> The issues as I see them:
>>
>> 1. Such buffers exist during the entire program's lifetime even if they
>> aren't actively used/needed by the program.  With malloc(3) and friends,
>> you're allocating memory dynamically, and you can free(3) when done with
>> it, rather than just having a gigantic portion of memory allocated
>> sitting around potentially doing nothing.
>>
>> 2. If the length of the buffer exceeds the amount of stack space
>> available at the time, the program will run but the behaviour is unknown
>> (I know that on FreeBSD if it exceeds "stacksize" per resource limits,
>> the program segfaults at runtime).  With malloc and friends you can
>> gracefully handle allocation failures.
>>
>> 3. Statically-allocated buffers can't grow; meaning what you've
>> requested size-wise is all you get.  Compare this to something that's
>> dynamic -- think a linked list containing pointers to malloc'd memory,
>> which can even be realloc(3)'d if needed.
>>
>> The definition of what's "too large" is up to the individual and the
>> limits of the underlying application.  For some people, sure, anything
>> larger than 2048 might warrant use of malloc.  I tend to use malloc for
>> anything larger than 4096.  That "magic number" comes from some piece of
>> information I was told long ago about what size pages malloc internally
>> uses, but looking at the IMPLEMENTATION NOTES section in malloc(3) it
>> appears to be a lot more complex than that.
>>
>> If you want me to break down #1 for you with some real-world output and
>> a very small C program, showing you the effects on RES/RSS and SIZE/VIRT
>> of static vs. dynamic allocation, just ask.
>>
> 
> Actually, after seeing that the userland limit for an unpriveleged user
> on freebsd is a mere 64k, I'd say the only valid reason to not allocate
> big things on the stack is because freebsd has completely broken
> defaults.  I see no reason why there should even be a distinction
> between stack size and memory use limits in general.  Pages are pages,
> it really doesn't matter what part of your virtual address space they
> live in.

I think that the stacksize and datasize cannot together add up to more
than 4G, or maybe it is only 2G, at least on a 32 bit machine.  This is
because (I think) they compete for virtual memory address space.  I
guess this wasn't a problem in the old days, when 4G of RAM was unthinkable.

Also, as I said in another thread, I think Linux doesn't make this
distinction.  So at least someone else out there agrees with you.