pthread_create() blocks if maxthreads reached?

[Craig Rodrigues]

On Sat, Jan 31, 2004 at 11:27:39PM -0800, Julian Elischer wrote:
> On Sun, 1 Feb 2004, Craig Rodrigues wrote:
> 
> > Hi,
> > 
> > I wrote the following small test program to spawn
> > a large number of threads.
> > 
> > ==================================================================
> > 
> > #include <sys/types.h>
> > #include <sys/select.h>
> > #include <pthread.h>
> > #include <stdio.h>
> > 
> > #define NUM 100000
> > 
> > void *thr(void *p)
> > {
> >   select(0, NULL, NULL, NULL, NULL);  
> >   return p;
> > }
> > 
> > 
> > int
> > main()
> > {
> >   pthread_t t[NUM];
> > 
> >   int ret=0;
> >   int i;
> >   
> >   for(i=0; ; ++i)
> >   {
> >      printf("Entering pthread_create...\n");
> >      ret = pthread_create(&t[i], NULL, thr, NULL);
> >      printf("Left pthread_create...\n");
> > 
> >      if( ret != 0 ) {
> >          printf("%d %s\n", ret, strerror(ret)); 
> >          break;
> >      }
> >   }
> >   printf("Max threads reached at: %d\n", i);
> > 
> >   return 0;
> > }
> > ==============================================================
> > 
> > 
> > If I link this program with -lthr or -lc_r, the
> > program will terminate.
> 
> Successfully? 
> that's interesting.. libthr should theoretically b elimited to about
> 8000 threads.. (on x86).

If I link with -lthr, I get:

Max threads reached at: 109


If I link with -lc_r, I get:

Max threads reached at: 14118


If I link with -lpthread, I get about 161 threads created,
and then pthread_create() blocks and I never get
to the "Max threads" message.



> 
> 
> > 
> > If I link with -lpthread, the program seems to
> > block in the pthread_create() call, instead of
> > returning EAGAIN (to indicate that a new thread cannot
> > be created due to hitting some system limit). 
> 
> is that what you expected?


No that is not what I expected.  I expected
that pthread_create() should either:
   - if it creates the thread successfully, return 0
   - if it cannot create the thread, return EAGAIN
     if a resource limit is it, or EINVAL if bad parameters
     were passed down into the pthread_create() call 


I never expected that pthread_create() would block.

> yes
> to stop processes running away and making the system unusable while we
> are developing the therad package we made a limit on how many 
> threads we allow to ente the kernel at one time per process..
> If you trust your program, you could over-ride the limit.
> 
> see:
> kern.threads.max_threads_per_proc: 150
> kern.threads.max_groups_per_proc: 50

Understood.  Keep in mind, I am trying to "kick the tires"
on libpthread, to see how it reacts on crazy corner cases.
I can certainly increase the values of these sysctls,
but I think that pthread_create() should not block if
it hits a limit.


> 
> Your threads are all waiting in the kernel.
> If you used a blocked semaphore or some other non-kernel
> method of blocking then you would discover that you could do more
> or you could allow more threads to enter the kernel at a time
> but that would be a poor use of resources.. it is much more
> efficient to keep blocked threads in userland than to block them in the
> kernel where they use precious kernel resources.
> I'm not sure but I think 'sleep()' is intecepted for example to 
> keep the sleeping threads in userland. (I may be wrong.
> I don't know if Dan did that yet).
> 
> 
> BTW the limit is not on making more threads, its that the last select()
> REALLY blocked and did not return to the userland program to allow you
> to make a new  thread.


Well, this may be so, but I don't think pthread_create() should block.
For kicks, I ran this code on Solaris 2.7, and pthread_create()
returned EAGAIN after spawning 3399 threads.


-- 
Craig Rodrigues        
http://crodrigues.org
rodrigc at crodrigues.org