getting stack trace for other thread on the same process : libthr

[Tijl Coosemans]

On Friday 26 September 2008 21:06:58 Dilip Chhetri wrote:
> Question
> --------
>    My program is linked with libthr in FreeBSD-7.0. The program has
> in the order of 20 threads, and a designated monitoring thread at
> some point wants to know what are other/stuck threads doing. This
> needs to be done by printing stack backtrace for the thread to
> stdout.
> 
>    I understand pthread_t structure has pointer to the target
> thread's stack, but to get the trace I need to know value of
> stack-pointer register and base-pointer register. I looked at the
> code and I don't find any mechanism by which I could read the target
> threads register context (because it all resides within kernel thread
> structure). Further code study reveals that kernel_thread->td_frame
> contains the register context for a thread, but is valid only when
> the thread is executing/sleeping inside the kernel.
> 
>    Is there anything I'm missing here ? Is there an easy way to
> traverse stack for some thread with in the same process.
> 
>    I considered/considering following approaches,
> a) use PTRACE
>     ruled out, because you can't trace the process from within the
>     same process
> 
> b) somehow temporarily stop the target-thread and read td_frame by
>     traversing kernel data structure through /dev/kmem. After doing
>     stack traversal resume the target thread.
> 
> 
> Detailed problem background
> --------------------------
>    We have this process X with ~20 threads, each processing some
> requests. One of them is designated as monitoring/dispatcher thread.
> When a new request arrives, dispatcher thread tries to queue the task
> to idle thread. But if all threads are busy processing requests, the
> dispatcher thread is supposed to print the stack back trace for each
> of the busy thread. This is our *debugging* mechanism to find
> potential fault-points.
> 
>    In FreeBSD-4.6.2, we hacked libc_r:pthread_t to achieve our goal.
> But in FreeBSD-7.0, we decided to use libthr and hack doesn't seem to
> be easy.
> 
> Target setup
> ------------
>   * SMP     : around 8 CPU
>   * process : it's going to be run as root and have around ~20 threads

You could try registering a signal handler for SIGUSR1 that prints a
stack backtrace using the stack pointer in the sigcontext and then call
pthread_kill(SIGUSR1) on whichever thread you want a backtrace of.