panic: in_pcblookup_local (?)

Glen Barber gjb at FreeBSD.org
Tue Apr 30 00:35:57 UTC 2013 

On Mon, Apr 29, 2013 at 12:24:06PM -0400, John Baldwin wrote:
> On Sunday, April 28, 2013 12:02:56 am Glen Barber wrote:
> > On Sat, Apr 27, 2013 at 10:17:32AM +0200, Ian FREISLICH wrote:
> > > Hi
> > > 
> > > I've been getting the following panic on recent current r249717.
> > > Sadly the crashdump is useless.
> > > 
> > 
> > I just saw similar panic on 10-CURRENT r249588.
> > 
> > > Fatal trap 9: general protection fault while in kernel mode
> > > cpuid = 15; apic id = 0f
> > > instruction pointer     = 0x20:0xffffffff80546fbc
> > > stack pointer           = 0x28:0xffffff846b677770
> > > frame pointer           = 0x28:0xffffff846b6777b0
> > > code segment            = base 0x0, limit 0xfffff, type 0x1b
> > >                         = DPL 0, pres 1, long 1, def32 0, gran 1
> > > processor eflags        = interrupt enabled, resume, IOPL = 0
> > > current process         = 4361 (zabbix_agentd)
> > 
> > Hmm..  This is interests me.  In my case, cf-agent was the current
> > process.
> > 
> > Backtrace of my panic follows.  Any pointers on how to debug this
> > further would be appreciated.
> > 
> > Glen
> > 
> > Script started on Sat Apr 27 23:53:53 2013
> > root at orion:/usr/obj/usr/src/sys/ORION # kgdb ./kernel.debug 
> /var/crash/vmcore.4
> > GNU gdb 6.1.1 [FreeBSD]
> > Copyright 2004 Free Software Foundation, Inc.
> > GDB is free software, covered by the GNU General Public License, and you are
> > welcome to change it and/or distribute copies of it under certain 
> conditions.
> > Type "show copying" to see the conditions.
> > There is absolutely no warranty for GDB.  Type "show warranty" for details.
> > This GDB was configured as "amd64-marcel-freebsd"...
> > 
> > Unread portion of the kernel message buffer:
> > 
> > 
> > Fatal trap 9: general protection fault while in kernel mode
> > cpuid = 1; apic id = 01
> > instruction pointer	= 0x20:0xffffffff80736cec
> > stack pointer	        = 0x28:0xffffff81aad4e760
> > frame pointer	        = 0x28:0xffffff81aad4e7a0
> > code segment		= base 0x0, limit 0xfffff, type 0x1b
> > 			= DPL 0, pres 1, long 1, def32 0, gran 1
> > processor eflags	= interrupt enabled, resume, IOPL = 0
> > current process		= 78664 (cf-agent)
> > trap number		= 9
> > panic: general protection fault
> > cpuid = 1
> > KDB: stack backtrace:
> > #0 0xffffffff80642a56 at kdb_backtrace+0x66
> > #1 0xffffffff80606eeb at panic+0x13b
> > #2 0xffffffff808e3b10 at trap_fatal+0x290
> > #3 0xffffffff808e4331 at trap+0x241
> > #4 0xffffffff808cdbb3 at calltrap+0x8
> > #5 0xffffffff807371d8 at in_pcb_lport+0x128
> > #6 0xffffffff8073745a at in_pcbbind_setup+0x16a
> > #7 0xffffffff80737d8e at in_pcbconnect_setup+0x71e
> > #8 0xffffffff80737df9 at in_pcbconnect_mbuf+0x59
> > #9 0xffffffff807bf29f at udp_connect+0x11f
> > #10 0xffffffff80680615 at kern_connectat+0x275
> > #11 0xffffffff80680731 at sys_connect+0x41
> > #12 0xffffffff808e32cb at amd64_syscall+0x63b
> > #13 0xffffffff808cde97 at Xfast_syscall+0xf7
> > Uptime: 3d19h38m52s
> > (ada0:ahcich0:0:0:0): FLUSHCACHE48. ACB: ea 00 00 00 00 40 00 00 00 00 00 00
> > (ada0:ahcich0:0:0:0): CAM status: CCB request is in progress
> > (ada0:ahcich0:0:0:0): Error 5, Retries exhausted
> > (ada0:ahcich0:0:0:0): Synchronize cache failed
> > (ada1:ahcich1:0:0:0): FLUSHCACHE48. ACB: ea 00 00 00 00 40 00 00 00 00 00 00
> > (ada1:ahcich1:0:0:0): CAM status: CCB request is in progress
> > (ada1:ahcich1:0:0:0): Error 5, Retries exhausted
> > (ada1:ahcich1:0:0:0): Synchronize cache failed
> > (ada2:ahcich4:0:0:0): FLUSHCACHE48. ACB: ea 00 00 00 00 40 00 00 00 00 00 00
> > (ada2:ahcich4:0:0:0): CAM status: CCB request is in progress
> > (ada2:ahcich4:0:0:0): Error 5, Retries exhausted
> > (ada2:ahcich4:0:0:0): Synchronize cache failed
> > (ada3:ahcich5:0:0:0): FLUSHCACHE48. ACB: ea 00 00 00 00 40 00 00 00 00 00 00
> > (ada3:ahcich5:0:0:0): CAM status: CCB request is in progress
> > (ada3:ahcich5:0:0:0): Error 5, Retries exhausted
> > (ada3:ahcich5:0:0:0): Synchronize cache failed
> > Dumping 1014 out of 6049 
> MB:..2%..12%..21%..32%..42%..51%..62%..71%..81%..92%
> > 
> > Reading symbols from /boot/kernel/zfs.ko.symbols...done.
> > Loaded symbols for /boot/kernel/zfs.ko.symbols
> > Reading symbols from /boot/kernel/opensolaris.ko.symbols...done.
> > Loaded symbols for /boot/kernel/opensolaris.ko.symbols
> > #0  doadump (textdump=<value optimized out>) at pcpu.h:231
> > 231		__asm("movq %%gs:%1,%0" : "=r" (td)
> > (kgdb) frame 6
> > #6  0xffffffff80736cec in in_pcblookup_local (pcbinfo=0xffffffff80dc9180, 
> laddr=
> >       {s_addr = 50374848}, lport=339, lookupflags=1, 
> cred=0xfffffe016cdad100)
> >     at /usr/src/sys/netinet/in_pcb.c:1438
> > 1438			LIST_FOREACH(phd, porthash, phd_hash) {
> > (kgdb) list *0xffffffff80736cec
> > 0xffffffff80736cec is in in_pcblookup_local 
> (/usr/src/sys/netinet/in_pcb.c:1439).
> > 1434			 * port hash list.
> > 1435			 */
> > 1436			porthash = &pcbinfo->ipi_porthashbase[INP_PCBPORTHASH(lport,
> > 1437			    pcbinfo->ipi_porthashmask)];
> > 1438			LIST_FOREACH(phd, porthash, phd_hash) {
> > 1439				if (phd->phd_port == lport)
> > 1440					break;
> > 1441			}
> > 1442			if (phd != NULL) {
> > 1443				/*
> 
> Can you see what 'phd' and 'porthash' are?  If kgdb can't see them you can
> reconstruct what 'porthash' should be (you know the lport arg to this routine
> and can get the relevant fields from 'pcbinfo').
> 

I'm not sure if the output included makes much sense.  If I did not do
something correctly, please let me know.

Glen

Script started on Mon Apr 29 20:27:39 2013
root at orion:/usr/obj/usr/src/sys/ORION #	kgdb ./kernel.debug /var/crash/vmcore.4
[...]
#0  doadump (textdump=<value optimized out>) at pcpu.h:231
231		__asm("movq %%gs:%1,%0" : "=r" (td)
(kgdb) frame 6
#6  0xffffffff80736cec in in_pcblookup_local (pcbinfo=0xffffffff80dc9180, laddr=
      {s_addr = 50374848}, lport=339, lookupflags=1, cred=0xfffffe016cdad100)
    at /usr/src/sys/netinet/in_pcb.c:1438
1438			LIST_FOREACH(phd, porthash, phd_hash) {
(kgdb) p *phd
Cannot access memory at address 0x9e17b100fffffe00
(kgdb) p *porthash
No symbol "porthash" in current context.
(kgdb) p &pcbinfo->ipi_porthashbase[INP_PCBPORTHASH(lport,pcbinfo->ipi_porthashmask)]
No symbol "INP_PCBPORTHASH" in current context.
(kgdb) p pcbinfo->ipi_porthashmask
$1 = 127
(kgdb) p *pcbinfo->ipi_porthashmask
Cannot access memory at address 0x7f
(kgdb) p *lport
Cannot access memory at address 0x153
(kgdb) p lport
$2 = 339
(kgdb) quit
root at orion:/usr/obj/usr/src/sys/ORION #	^D

Script done on Mon Apr 29 20:32:01 2013


-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-current/attachments/20130429/c617d07a/attachment.sig>

More information about the freebsd-current mailing list