CTM problem

[Peter Jeremy]

Hi Stephen,

On Mon, Jan 19, 2004 at 09:03:24PM -0600, Stephen Montgomery-Smith wrote:
>The problem is when mkctm is comparing the file ports/INDEX,v in the old 
>and new directories.  These files are getting to about 60 to 70MB in size.  
>mkctm contains a function called Equ.  First Equ does a mmap to map each of 
>the files to memory, so that it can compare them to see if they are the 
>same size.  If they are different, Equ then attempts to use alloca to 
>allocate memory of the same size as the larger of the two files (plus 2).  
>This memory is then used to store the output of "diff -n".  It is during 
>this call to alloca that mkctm has a segmentation fault.
>
>Thus the program needs to allocate about 3 times 60 or 70MB.  I am not at 
>all familiar with how FreeBSD allocates memory.  I see from the man page 
>that alloca gets its memory from the stack.  I am guessing that mmap gets 
>its memory from the heap, but I am not sure.

alloca() does allocate from the stack.  mmap() allocates virtual space
but not from the heap.

>I remember working with these kinds of problems before.  I know that there 
>is a command to change how much space processes are allowed to use, but I 
>don't remember the commands for this right now.  Maybe someone could remind 
>me what they are, and in that case maybe just increasing the stack allowed 
>by processes would fix the problem.

You are thinking of either ulimit(1) or setrlimit(2).  Unfortunately,
this isn't going to be enough in this case.  The default stack size is
set to the maximum - 65536KB for the i386.  To increase this, you will
need to set kern.maxssiz at boot time.  (Though you can reduce it via
ulimit, setrlimit or /etc/login.conf).  Add a line like
"kern.maxssiz=131072" to /boot/loader.conf and reboot to double the
default stack size.

I presume the machine you're running mkctm on has plenty of RAM - from
a quick look at the mkctm code, I wouldn't suggest running it in any
less than 256MB RAM - preferably 512MB.

Peter