seg fault on kse_release () (fwd)

[Peter Edwards]

On Mon, 24 Jan 2005 14:27:53 -0800 (PST), Yan Yu <yanyu at cs.ucla.edu> wrote:
> Hi, all,  I have a newbie Q:
>     I am trying to use creating large number of threads and allocting
> memory to stress the system.
> My user program causes SEG fault in the kernel code, kse_release () in
> kern_kse.c.
> (it SEG fault before the system can be stressed;(
> 
> the stack when the SEG fault happens are:
> #0  0x08064e54 in kse_release ()
> #1  0x080531c4 in kse_sched_single ()
> #2  0x00000000 in ?? ()
> 
> My simple program is:
> I have a simple function to create threads:
> 
> #define NUM_THREADS     5000
> #define THREADS_IN_ONE_PROCESS  5
> #define BSIZE  500000
> static int  cc;
> 
> void CreateThread(int n)
> {
>    assert( n <= NUM_THREADS );
>    pthread_t threads[NUM_THREADS];
>    int rc, t;
>    for(t=0;t < n;t++){
>       printf("#%d: Creating thread %d\n", cc, t);
>       cc++;
>       rc = pthread_create(&threads[t], NULL, PrintHello, (void *)t);
>       if (rc){
>          printf("ERROR; return code from pthread_create() is %d\n", rc);
>       }
>    }
> 
>    unsigned long id;
>    char * p = (char *) calloc(BSIZE, sizeof(char) );
>    if ( p == NULL )
>    {
>         fprintf(stderr, "calloc error\n");
>    }
>    while (1)
>    {
>         while (BSIZE <= (id = rand() / (RAND_MAX/BSIZE)));
>         p[id] ++;
>    }
> }
> 
> void *PrintHello(void *threadid)
> {
>    printf("\n%d: Hello World!\n", threadid);
>    CreateThread(THREADS_IN_ONE_PROCESS);
>    pthread_exit(NULL);
> }
> 
> int main (int argc, char *argv[])
> {
>    CreateThread(THREADS_IN_ONE_PROCESS);
> }
> 
> The SEG fault happens after creating nearly 5000 threads.
> and I use the default pthread.h coming w/ freeBSD 5.3
> #define PTHREAD_KEYS_MAX                        256
> #define PTHREAD_STACK_MIN                       (1 << 22)
> #define PTHREAD_THREADS_MAX                     ULONG_MAX
> 
> Any idea on what might happen?
> 
> Many Thanks!
> yan

A few things:

kse_release() in this context isn't kernel code, its from libpthread.
The kernel equivalent of a segfault would generate a panic.

Just in case: you're attempting to allocate more than 5000 threads:
Each one from main tries to recursively create 5000 more, and so on.
Also note that because you are not calling pthread_join() (and aren't
creating "detatched" threads) you're leaking memory as each thread
exits.

Now, if it's actually dying on the 5000ish thread mark:

Also, each thread will allocate a stack + some extra overhead. The
default stack size is 64K. That means you're allocating 312MB of stack
space, and 5000 x some constant on top of that. That might exceed your
process's data segment size (though I'm not sure that limit is imposed
entirely by FreeBSDs malloc: I suspect it isn't)

It's certainly a lot of memory for a 32-bit process. You could try
increasing your data segment size with ulimit(1), or decreasing the
stack size per thread with
pthread_attr_init/pthread_attr_setstacksize().

Also: gdb for corefiles may have been broken in 5.3, showing the
backtrace of the wrong stack. (I fixed one problem post release, but
I'm not sure if it was introduced before or after. Can you either run
the program from gdb (so you debug a live process rather than a
corefile), or try updating gdb?