Trouble with NFSd under 6.1-Stable, any ideas?

[Howard Leadmon]

   Hello Rong-en,

 Thanks for the info on getting the debugger configured, and on the serial
console.   I will have to try and play with the serial console thing more, I
just tried putting in the flags and the damn thing hung, I had to boot from CD
and take the stuff back out.

 One thing you mention below that concerns me is that you have version 1.90 of
the vfs_lookup.c file.   I just did a less on /usr/src/sys/kern/vfs_lookup.c
and I see the following:

FreeBSD: src/sys/kern/vfs_lookup.c,v 1.80.2.7 2006/04/30 03:57:46 kris Exp


 I even did a cvsup (I use cvsup2.FreeBSD.org) to make sure I had the current
stuff before rebuilding the kernel just now, and still I see the same thing.
Is something fishy going on here, or did you by chance make a typo??


---
Howard Leadmon - howard at leadmon.net
http://www.leadmon.net

 

> -----Original Message-----
> From: Rong-en Fan [mailto:grafan at gmail.com] 
> Sent: Tuesday, May 23, 2006 10:19 AM
> To: Howard Leadmon
> Cc: Konstantin Belousov; Kris Kennaway; freebsd-stable at freebsd.org
> Subject: Re: Trouble with NFSd under 6.1-Stable, any ideas?
> 
> On 5/23/06, Howard Leadmon <howard at leadmon.net> wrote:
> >
> >
> > > > > If there are any thing I can provide to help tracking 
> this down.
> > > > > Please let me know. By the way, I tried with truss/kdump
> > > to see what
> > > > > happens when nfsd eats lot of CPUs, but in vain. They do
> > > not return anything.
> > > > >
> > > > I tried your recipe on 7-CURRENT with locally exported fs,
> > > remounted
> > > > over nfs. I did not get the behaviour your described.
> > >
> > > As noted in my previous thread, I have another 6.1-RELEASE nfs 
> > > server, which does not have this problem.
> > >
> > > > Could you, please, provide the backtrace for the nfsd that eats 
> > > > the CPU (from the ddb). I think it would be helpful to 
> get several 
> > > > backtraces (i.e., bt <nfsd pid>, cont, bt <nfsd pid> ...)
> > > to see where
> > > > it running.
> > >
> > > I'm afraid that I can not do that. Last time I tried 
> breaking into 
> > > ddb (on 5.x), it hangs my serial console and the server is miles 
> > > away :-( . Perhaps we can ask Howard to do that?
> >
> >  I am more than willing to do that, as this machine runs 
> here with me, 
> > so if needed I can easily get on a console, or perform a 
> reboot.  Can 
> > one of you shed a little light on exactly what I need to 
> do, and how 
> > to do this?  I ask as I have never used this ddb stuff, so not clue 
> > one on how to go about getting the information your looking 
> to find.  
> > Guess I have been lucky, and just never had an issue that 
> took things to this level.
> 
> At least you have to add the following to your kernel:
> 
> options         KDB
> options         DDB
> 
> Recompile it, reboot. You would better to setup a serial 
> console so you can easily copy thing from ddb output. To do 
> it, you have to put "device sio" in your kernel configuration 
> and some files
> below:
> 
> /boot.config
> -Dh
> 
> /boot/loader.conf
> comconsole_speed=115200
> machdep.conspeed=115200
> 
> /etc/ttys
> ttyd0   "/usr/libexec/getty std.115200" cons25  on secure
> 
> On the other machine, /etc/remote:
> com1:dv=/dev/cuad0:br#115200:pa=none:
> 
> Then, use "tip com1" to attach the nfs server. The above 
> settings assume your serial console on nfs server is on COM1 
> and on the client side is also COM1. If that's not the case, 
> please follow Handbook for howto setup a serial console other 
> than COM1. To break into ddb, either use ctrl+alt+esc or send 
> a BREAK (I think ^b will do) via serial line. After that, you 
> should see
> 
> db>
> 
> Then you first use "ps" to find out the nfsd pid (better to 
> remember the pid which eats  lots of cpu before enter ddb). 
> After that, do what Konstantin suggests. I have never tried 
> "cont" in db. I guess that will return the execution back to 
> kernel and you need to break into ddb again to do another "bt <pid>".
> 
> By the way, could you verify that backing out vfs_lookup.c 
> rev 1.90 helps in your situation? If not, maybe we are seeing 
> different problems, and then I have to figure out how to make 
> my serial console work here.
> 
> Thanks,
> Rong-En Fan
>