How to debug rc hangs?

[John Marshall]

Mike Telahun Makonnen wrote:
> On 10/23/07, John Marshall <John.Marshall at riverwillow.com.au> wrote:
>> I have tried setting rc_debug="YES" in rc.conf but that doesn't show me
>> any more than I already know (e.g. last line before mountd hang is:
>> "/etc/rc: DEBUG: run_rc_command: doit: /usr/sbin/mountd -l"
> 
> It seems to me that if it's getting this far, that the problem probably is
> not in rc.d. The next thing it does after that debug message is eval the $doit
> line you saw, so either the eval command is missbehaving or the problem
> is with the daemon and not rc.d. What does CTR-t say when it hangs? Also,
> I noticed all three programs you listed are network daemons. My guess is
> they are not actually hung, they only *appear* to hang because they're wating
> on some sort of network resource (DNS maybe?).
> 

Thanks Mike,

The ctrl-T tip is the kind of information I'm looking for. My primary 
reason for posting is to find out what tools/switches/hooks are 
available to help troubleshoot this kind of problem, rather than asking 
somebody else to solve it.

Having said that, ctrl-T shows:
  load: 0.74 cmd: mountd 576 [nanslp] 0.00u 0.00s 0% 1428k
  load: 0.25 cmd: mountd 576 [nanslp] 0.00u 0.00s 0% 1432k
  load: 0.12 cmd: mountd 576 [nanslp] 0.00u 0.00s 0% 1432k

...which lends weight to my suspicion that a pre-requisite resource is 
not yet available - and, perhaps, hasn't yet started due to a circular 
dependency? As I hinted, my plan is to drill down into the 
PROVIDE/REQUIRE labyrinth and work by trial and error (with a reboot in 
between each error). I'm happy to do that but I'm hoping that I might be 
able to use this situation to learn of more elegant ways to diagnose the 
problem.

...and to reiterate, this is on 7.0-BETA1 (built Saturday morning) and 
all this was working without any intervention on 6.2-RELEASE.


-- 
John Marshall