getting stack trace for other thread on the same process : libthr

[Dilip Chhetri]

Question
--------
   My program is linked with libthr in FreeBSD-7.0. The program has in 
the order of 20 threads, and a designated monitoring thread at some 
point wants to know what are other/stuck threads doing. This needs to be 
done by printing stack backtrace for the thread to stdout.

   I understand pthread_t structure has pointer to the target thread's 
stack, but to get the trace I need to know value of stack-pointer 
register and base-pointer register. I looked at the code and I don't 
find any mechanism by which I could read the target threads register 
context (because it all resides within kernel thread structure). Further 
code study reveals that kernel_thread->td_frame contains the register 
context for a thread, but is valid only when the thread is 
executing/sleeping inside the kernel.

   Is there anything I'm missing here ? Is there an easy way to traverse 
stack for some thread with in the same process.

   I considered/considering following approaches,
a) use PTRACE
    ruled out, because you can't trace the process from within the
    same process

b) somehow temporarily stop the target-thread and read td_frame by
    traversing kernel data structure through /dev/kmem. After doing
    stack traversal resume the target thread.


Detailed problem background
--------------------------
   We have this process X with ~20 threads, each processing some 
requests. One of them is designated as monitoring/dispatcher thread. 
When a new request arrives, dispatcher thread tries to queue the task to 
idle thread. But if all threads are busy processing requests, the 
dispatcher thread is supposed to print the stack back trace for each of 
the busy thread. This is our *debugging* mechanism to find potential 
fault-points.

   In FreeBSD-4.6.2, we hacked libc_r:pthread_t to achieve our goal. But 
in FreeBSD-7.0, we decided to use libthr and hack doesn't seem to be easy.

Target setup
------------
  * SMP     : around 8 CPU
  * process : it's going to be run as root and have around ~20 threads