[ace-users] ace/freebsd: THR_NEW_LWP problem with libpthread/pthread_setconcurrency

[Andriy Gapon]

ACE VERSION:
5.5.1 (from FreeBSD port: ace-5.5.1)

HOST MACHINE and OPERATING SYSTEM:
FreeBSD 6.1-RELEASE-p2 i386 (uniprocessor)

COMPILER NAME AND VERSION (AND PATCHLEVEL):
gcc version 3.4.4 [FreeBSD] 20050518 (system compiler)
libpthread is used as threading library

AREA/CLASS/EXAMPLE AFFECTED:
ACE threads

DOES THE PROBLEM AFFECT:
EXECUTION - this is a run-time problem

SYNOPSIS:
Can not create threads with THR_NEW_LWP flag.

DESCRIPTION:
ACE_Thread_Manager::spawn(..., THR_NEW_LWP) fails with EAGAIN. It seems
that this happens because when creating such threads ACE calls
pthread_setconcurrency with incrementally increasing concurrency levels.
pthread_setconcurrency of libpthread in turn seems to call kse_create()
which fails with EPROCLIM. As I understand this happens because kernel
does not allow to have more KSE-s per process as there are processors.

This problem is quite noticeable with ACE because THR_NEW_LWP is present
in default flags in many places.

REPEAT BY:
Running any ACE-based program that spawns threads with THR_NEW_LWP and
linked to libpthead.

SAMPLE FIX/WORKAROUND:
Straightforward workaround is to explicitly pass thread flags everywhere
and not use THR_NEW_LWP. Maybe this could be patched in ACE sources as a
part of FreeBSD port patch step.
IMHO, better fixes would be:
1. make ACE_Thread_Manager::spawn() more robust with respect to
pthread_setconcurrency() failing with EAGAIN.
2. make FreeBSD libpthread dumb-happy in pthread_setconcurrency(), i.e
pretend to always succeed. AFAIK, POSIX leaves it up to implementations
to interpret concurrency levels, so making any level be equivalent to
default level should be OK.

-- 
Andriy Gapon