Process Debugging questions

[Robert Watson]

On Tue, 21 Nov 2006, Dag-Erling Smørgrav wrote:

> Jeremie Le Hen <jeremie at le-hen.org> writes:
>> Dag-Erling Smørgrav <des at des.no> writes:
>>> They both suck, for different reasons.  In theory, ptrace sucks less than 
>>> proc, but it lacks some of proc's functionality, and fixing that is very 
>>> hard.
>> Would you take a little time to tell what ptrace lacks and possibly why it 
>> is so hard, please ?
>
> The way ptrace works, you set debugging conditions using the ptrace(2) 
> syscall and then use waitpid(2) and friends to wait for them to occur. For 
> this to work, the traced process must be reparented to the debugger.  If the 
> traced process's real parent is waiting for its child, it will become very 
> confused when waitpid(2) returns -1 because the child has vanished into thin 
> air.  For precisely the same reason, you can't follow forks with ptrace().
>
> The only way I can see to solve this without modifying the ptrace API is to 
> introduce a separate process hierarchy for traced processes. This is hard to 
> do because you basically have to rewrite kern_wait() from scratch.
>
> The best solution would be to design a new debugging API from scratch. This 
> is far from trivial, however, and should be done by (or in close cooperation 
> with) someone intimately familiar with gdb(1) and similar tools.

DES and I had precisely this discussion at EuroBSDCon a week or two ago, in 
which we both reminded each other why ptrace() and procfs both suck.  I think 
the right answer is an improved system call API for debugging; I agree that 
ptrace() is better in theory, but the practical problems with it are 
significant.

Anyone who wants to start looking at a new API should make sure they look at 
ptrace() and procfs across a range of platforms, and also look at Mach task 
ports and what they offer.  One of the things we really can't do well with 
ptrace() today is inspect kernel state for a running process -- things like 
the VM layout, etc.  We should make sure any next generation system allows 
this, because that feature alone is the reason why I sometimes mount procfs. 
:-)

Robert N M Watson
Computer Laboratory
University of Cambridge