mysterious hang in pthread_create

[Daniel Eischen]

On Sat, 30 Aug 2008, Kostik Belousov wrote:

> On Sat, Aug 30, 2008 at 12:15:31PM -0400, Daniel Eischen wrote:
>> On Sat, 30 Aug 2008, Kostik Belousov wrote:
>>
>>> On Sat, Aug 30, 2008 at 11:32:35AM -0400, Daniel Eischen wrote:
>>>> On Fri, 29 Aug 2008, Kostik Belousov wrote:
>>>>>
>>>>> As demonstrated by Andriy' example, we need _thr_rtld_init() be called
>>>>> before any rtld locks are given chance to be acquired. _thr_rtld_init()
>>>>> shall be protected from repeated invocation, and _thr_setthreaded()
>>>>> implements exactly this.
>>>>>
>>>>> If calling _thr_setthreaded(1) has not quite right intent, could you,
>>>>> please, suggest satisfying solution ?
>>>>
>>>> I'm not sure I _quite_ understand the problem, but why
>>>> wouldn't you have the same potential problem with some
>>>> other library (without libthread)?  I'll have to go back
>>>> and read the beginning of the thread - I just kinda came
>>>> in at the end.
>>>
>>> Sure, for appropriate value of any. If you mean whether the same problem
>>> would arise for any threading library that supplies locking implementation
>>> for rtld, then certainly yes. I looked over and patched only libthr
>>> since this is the only survived library for now.
>>
>> What I mean is, is fixing libthr a solution that will work
>> for cases?  Or, is libthr doing something wrong?  I can't
>> really see that it is.
>>
>> libthr assumes that everything is single-threaded (or
>> serialized, I guess) before a thread is created.  I
>> am looking at this thread:
>>
>>   http://docs.freebsd.org/cgi/getmsg.cgi?fetch=5235+0+current/freebsd-threads
>>
>> Where is the corresponding unlock for the wlock_acquire()?
>> I guess this is the problem.  When would this normally
>> be released (without libthr being linked in)?
>>
>> Also, the __isthreaded flag is used in libc to avoid taking
>> locks unless necessary.  So if you have a single threaded
>> application that is also linked with libthr, you don't
>> pay the penalty of locking overhead.  Lots of 3rd-party
>> libraries link with a threads library, so an application
>> may not even know it is "threaded".
>>
>>>
>>> Anyway, I do not insist on the proposed solution, and definitely
>>> prefer the change that is well aligned with libthr architecture.
>>
>> I'm not arguing anything, I just don't know that the problem
>> lies within lib<insert thread library here>.  Of course, the
>> rtld init stuff could be pulled out and done in thread
>> initialization instead of thr_setthreaded().  That doesn't
>> leave much in thr_setthreaded, and it also adds locking
>> overhead into rtld for single-threaded programs that are
>> linked with libthr...
>
> Ok, let me to tell the whole story. I am sure that in fact you know
> it better then me.
>
> Assuming libthr is the only threading library, there are two locking
> implementations for the rtld: 'default' and the one supplied by libthr.
> On the first call to pthread_create(), libthr calls _rtld_thread_init()
> to substitute the default by the implementation from libthr.
>
> In fact, default implementation is broken from my point of view. For
> instance, thread_flag update is not atomic. Moreover, it does not
> correctly handles sequential acquision of several locks, due
> to thread_flag.
>
> The dl_iterate_phdr() function, called by gcc exception handling support
> code, does exactly this. It acquires rtld_phdr_lock, then rtld_bind_lock.
> [I shall admit it does this after my change]. In particular, this would
> leave the bit for the bind lock set in the thread_flag.
>
> Andriy' example throw the exception and calls dl_iterate_phdr() before
> first thread is created. On thread creation, _rtld_thread_init() is
> called, that tries to move the locks according to thread_flag. This is
> the cause for the reported wlock acquisition.
>
> I do not want to change anything in the default rtld locking. It is
> disfunctional from the time libc_r is gone, and I think it would be
> better to make it nop. My change makes the image that is linked with
> libthr, to consistently use libthr locks.

What happens if you remove the thread_flag() stuff (support
for libc_r?) from rtld?  It seems that libc_r should be providing
its own rtld locking hooks - just like libthr does.

-- 
DE