getting stack trace for other thread on the same process : libthr

[Dilip Chhetri]

Tijl Coosemans wrote:
> On Friday 26 September 2008 21:06:58 Dilip Chhetri wrote:
> 
>>Question
>>--------
>>   My program is linked with libthr in FreeBSD-7.0. The program has
>>in the order of 20 threads, and a designated monitoring thread at
>>some point wants to know what are other/stuck threads doing. This
>>needs to be done by printing stack backtrace for the thread to
>>stdout.
>>
>>   I understand pthread_t structure has pointer to the target
>>thread's stack, but to get the trace I need to know value of
>>stack-pointer register and base-pointer register. I looked at the
>>code and I don't find any mechanism by which I could read the target
>>threads register context (because it all resides within kernel thread
>>structure). Further code study reveals that kernel_thread->td_frame
>>contains the register context for a thread, but is valid only when
>>the thread is executing/sleeping inside the kernel.
>>
>>   Is there anything I'm missing here ? Is there an easy way to
>>traverse stack for some thread with in the same process.
>>
>>   I considered/considering following approaches,
>>a) use PTRACE
>>    ruled out, because you can't trace the process from within the
>>    same process
>>
>>b) somehow temporarily stop the target-thread and read td_frame by
>>    traversing kernel data structure through /dev/kmem. After doing
>>    stack traversal resume the target thread.
>>
>>
>>Detailed problem background
>>--------------------------
>>   We have this process X with ~20 threads, each processing some
>>requests. One of them is designated as monitoring/dispatcher thread.
>>When a new request arrives, dispatcher thread tries to queue the task
>>to idle thread. But if all threads are busy processing requests, the
>>dispatcher thread is supposed to print the stack back trace for each
>>of the busy thread. This is our *debugging* mechanism to find
>>potential fault-points.
>>
>>   In FreeBSD-4.6.2, we hacked libc_r:pthread_t to achieve our goal.
>>But in FreeBSD-7.0, we decided to use libthr and hack doesn't seem to
>>be easy.
>>
>>Target setup
>>------------
>>  * SMP     : around 8 CPU
>>  * process : it's going to be run as root and have around ~20 threads
> 
> 
> You could try registering a signal handler for SIGUSR1 that prints a
> stack backtrace using the stack pointer in the sigcontext and then call
> pthread_kill(SIGUSR1) on whichever thread you want a backtrace of.

Thanks, but as I mentioned it's a network based program and it may be 
sleeping/stuck in syscall for some packets, in this case pthread_kill 
will not work because signals are delivered only when you return from 
syscall (that's what I haved learned from old UNIX books in my college).