Re: FreeBSD panics possibly caused by nfs clients
Date: Mon, 19 Feb 2024 22:50:19 UTC
On Mon, Feb 19, 2024 at 7:44 AM Matthew L. Dailey <Matthew.L.Dailey@dartmouth.edu> wrote: > > Hi all, > > So I finally induced a panic on a "pure" ufs system - root and exported > filesystem were both ufs. So, I think this definitively rules out zfs as > a source of the issue. > > This panic was on 14.0p5 without debugging options, so the core may not > be helpful. The panic and backtrace are below in case they're > interesting to anyone. > > Next, I'm going to try a CURRENT kernel without debugging options > enabled just to see if I can finally induce a panic here. My other two > VMs running CURRENT with full debugging are still clanking along. You might want to set kern.kstack_pages=6 in /boot/loader.conf in these setups. I would normally expect double faults when a kernel stack is blown, but maybe there is a reason that you do now see that for a blown kernel stack. (The impact of increasing stack pages from 4->6 should be minimal.) rick > > -Matt > > [218716] Fatal trap 12: page fault while in kernel mode > [218716] cpuid = 4; apic id = 08 > [218716] fault virtual address = 0x100000017 > [218716] fault code = supervisor read data, page not present > [218716] instruction pointer = 0x20:0xffffffff80e9165d > [218716] stack pointer = 0x28:0xfffffe010b5aa3b0 > [218716] frame pointer = 0x28:0xfffffe010b5aa400 > [218716] code segment = base 0x0, limit 0xfffff, type 0x1b > [218716] = DPL 0, pres 1, long 1, def32 0, gran 1 > [218716] processor eflags = interrupt enabled, resume, IOPL = 0 > [218716] current process = 49575 (nfsd: service) > [218716] rdi: 00000000ffffffff rsi: fffff800038ec900 rdx: fffffe00d9326000 > [218716] rcx: 0000000000030eb0 r8: 0000000000000000 r9: fffffe010b5aa410 > [218716] rax: 00000000008f0eb0 rbx: fffff8038ac4cd00 rbp: fffffe010b5aa400 > [218716] r10: 0000000000000000 r11: 0000000000000000 r12: 0000000000000000 > [218716] r13: fffff80003647c00 r14: fffff802f9dced00 r15: fffff800038ec900 > [218716] trap number = 12 > [218716] panic: page fault > [218716] cpuid = 4 > [218716] time = 1708319487 > [218716] KDB: stack backtrace: > [218716] #0 0xffffffff80b9309d at kdb_backtrace+0x5d > [218716] #1 0xffffffff80b461a2 at vpanic+0x132 > [218716] #2 0xffffffff80b46063 at panic+0x43 > [218716] #3 0xffffffff8101d85c at trap_fatal+0x40c > [218716] #4 0xffffffff8101d8af at trap_pfault+0x4f > [218716] #5 0xffffffff80ff3fe8 at calltrap+0x8 > [218716] #6 0xffffffff80e8716e at newdirrem+0x8be > [218716] #7 0xffffffff80e866fa at softdep_setup_remove+0x1a > [218716] #8 0xffffffff80ea71af at ufs_dirremove+0x21f > [218716] #9 0xffffffff80ead4f4 at ufs_remove+0xb4 > [218716] #10 0xffffffff810f1428 at VOP_REMOVE_APV+0x28 > [218716] #11 0xffffffff80a60db4 at nfsvno_removesub+0xc4 > [218716] #12 0xffffffff80a52699 at nfsrvd_remove+0x1b9 > [218716] #13 0xffffffff80a374d4 at nfsrvd_dorpc+0x1854 > [218716] #14 0xffffffff80a4e76f at nfssvc_program+0x82f > [218716] #15 0xffffffff80e34080 at svc_run_internal+0xb50 > [218716] #16 0xffffffff80e3475b at svc_thread_start+0xb > [218716] #17 0xffffffff80b00b7f at fork_exit+0x7f > [218716] Uptime: 2d12h45m16s > [218716] Dumping 985 out of 16350 > MB:..2%..12%..22%..31%..41%..51%..61%..72%..82%..91% > > > #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 > #1 doadump (textdump=<optimized out>) at > /usr/src/sys/kern/kern_shutdown.c:405 > #2 0xffffffff80b45d37 in kern_reboot (howto=260) > at /usr/src/sys/kern/kern_shutdown.c:526 > #3 0xffffffff80b4620f in vpanic (fmt=0xffffffff81147c9c "%s", > ap=ap@entry=0xfffffe010b5aa200) at > /usr/src/sys/kern/kern_shutdown.c:970 > #4 0xffffffff80b46063 in panic (fmt=<unavailable>) > at /usr/src/sys/kern/kern_shutdown.c:894 > #5 0xffffffff8101d85c in trap_fatal (frame=0xfffffe010b5aa2f0, > eva=4294967319) > at /usr/src/sys/amd64/amd64/trap.c:952 > #6 0xffffffff8101d8af in trap_pfault (frame=0xfffffe010b5aa2f0, > usermode=false, signo=<optimized out>, ucode=<optimized out>) > at /usr/src/sys/amd64/amd64/trap.c:760 > #7 <signal handler called> > #8 cancel_diradd (dap=0xfffff8038ac4cd00, > dirrem=dirrem@entry=0xfffff800038ec900, > jremref=jremref@entry=0xfffff802f9dced00, > dotremref=dotremref@entry=0x0, > dotdotremref=dotdotremref@entry=0x0) > at /usr/src/sys/ufs/ffs/ffs_softdep.c:9028 > #9 0xffffffff80e8716e in newdirrem (bp=<optimized out>, > dp=dp@entry=0xfffff800037fea80, ip=ip@entry=0xfffff8006b3b9300, > isrmdir=isrmdir@entry=0, > prevdirremp=prevdirremp@entry=0xfffffe010b5aa4b0) > at /usr/src/sys/ufs/ffs/ffs_softdep.c:9480 > #10 0xffffffff80e866fa in softdep_setup_remove (bp=0xffffffff, > dp=0xfffff800038ec900, dp@entry=0xfffff800037fea80, > ip=0xfffffe00d9326000, > ip@entry=0xfffff8006b3b9300, isrmdir=200368, isrmdir@entry=0) > at /usr/src/sys/ufs/ffs/ffs_softdep.c:9176 > #11 0xffffffff80ea71af in ufs_dirremove (dvp=dvp@entry=0xfffff801f764be00, > ip=ip@entry=0xfffff8006b3b9300, flags=<optimized out>, > isrmdir=isrmdir@entry=0) at /usr/src/sys/ufs/ufs/ufs_lookup.c:1198 > #12 0xffffffff80ead4f4 in ufs_remove (ap=0xfffffe010b5aa5d8) > at /usr/src/sys/ufs/ufs/ufs_vnops.c:1054 > #13 0xffffffff810f1428 in VOP_REMOVE_APV ( > vop=0xffffffff8172f2d0 <ffs_vnodeops2>, a=a@entry=0xfffffe010b5aa5d8) > at vnode_if.c:1534 > #14 0xffffffff80a60db4 in VOP_REMOVE (dvp=0x8f0eb0, vp=0xfffff800539b7380, > cnp=0x30eb0) at ./vnode_if.h:789 > #15 nfsvno_removesub (ndp=0xfffffe010b5aa858, is_v4=<optimized out>, > cred=<optimized out>, p=p@entry=0xfffffe010ae803a0, > exp=exp@entry=0xfffffe010b5aaa88) > at /usr/src/sys/fs/nfsserver/nfs_nfsdport.c:1466 > #16 0xffffffff80a52699 in nfsrvd_remove (nd=0xfffffe010b5aabf8, > isdgram=<optimized out>, dp=<optimized out>, exp=0xfffffe010b5aaa88) > at /usr/src/sys/fs/nfsserver/nfs_nfsdserv.c:1583 > #17 0xffffffff80a374d4 in nfsrvd_compound (nd=0xfffffe010b5aabf8, isdgram=0, > tag=<optimized out>, taglen=0, minorvers=<optimized out>) > at /usr/src/sys/fs/nfsserver/nfs_nfsdsocket.c:1322 > #18 nfsrvd_dorpc (nd=nd@entry=0xfffffe010b5aabf8, isdgram=isdgram@entry=0, > tag=<optimized out>, taglen=0, minorvers=<optimized out>) > at /usr/src/sys/fs/nfsserver/nfs_nfsdsocket.c:633 > #19 0xffffffff80a4e76f in nfs_proc (nd=0xfffffe010b5aabf8, > xid=<optimized out>, xprt=0xfffff800037b4800, rpp=<optimized out>) > at /usr/src/sys/fs/nfsserver/nfs_nfsdkrpc.c:464 > #20 nfssvc_program (rqst=0xfffff80128ac9000, xprt=0xfffff800037b4800) > at /usr/src/sys/fs/nfsserver/nfs_nfsdkrpc.c:348 > #21 0xffffffff80e34080 in svc_executereq (rqstp=0xfffff80128ac9000) > at /usr/src/sys/rpc/svc.c:1036 > #22 svc_run_internal (grp=grp@entry=0xfffff800033c2100, > ismaster=ismaster@entry=0) at /usr/src/sys/rpc/svc.c:1312 > #23 0xffffffff80e3475b in svc_thread_start (arg=0xffffffff, > arg@entry=0xfffff800033c2100) at /usr/src/sys/rpc/svc.c:1340 > #24 0xffffffff80b00b7f in fork_exit ( > callout=0xffffffff80e34750 <svc_thread_start>, arg=0xfffff800033c2100, > frame=0xfffffe010b5aaf40) at /usr/src/sys/kern/kern_fork.c:1160 > #25 <signal handler called> > #26 0x000033af7af143ea in ?? ()