random ZFS panic...

Larry Rosenman ler at lerctr.org
Wed Mar 2 12:18:23 UTC 2016 

Was rebooting my laptop, and got the following:


trivet dumped core - see /var/crash/vmcore.0

Wed Mar  2 06:09:19 CST 2016

FreeBSD trivet 11.0-CURRENT FreeBSD 11.0-CURRENT #3 r296287: Tue Mar  1 19:13:37 CST 2016     root at trivet:/usr/obj/usr/src/sys/GENERIC  amd64

panic: from debugger

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:
<118>.
<118>Terminated
<118>Mar  2 06:07:22 trivet syslogd: exiting on signal 15
<5>wlan0: link state changed to DOWN
Waiting (max 60 seconds) for system process `vnlru' to stop...done
Waiting (max 60 seconds) for system process `bufdaemon' to stop...done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining...0 0 0 0 done
All buffers synced.
lock order reversal:
 1st 0xfffff80012855d50 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:1222
 2nd 0xfffff800128557c8 zfs_gfs (zfs_gfs) @ /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/gfs.c:494
stack backtrace:
#0 0xffffffff80a825a0 at witness_debugger+0x70
#1 0xffffffff80a824a1 at witness_checkorder+0xe71
#2 0xffffffff80a00a6b at __lockmgr_args+0xd3b
#3 0xffffffff80ace2bc at vop_stdlock+0x3c
#4 0xffffffff80fcb010 at VOP_LOCK1_APV+0x100
#5 0xffffffff80aeef9a at _vn_lock+0x9a
#6 0xffffffff820c8b13 at gfs_file_create+0x73
#7 0xffffffff820c8bbd at gfs_dir_create+0x1d
#8 0xffffffff82191f57 at zfsctl_mknode_snapdir+0x47
#9 0xffffffff820c9135 at gfs_dir_lookup+0x185
#10 0xffffffff820c961d at gfs_vop_lookup+0x1d
#11 0xffffffff82190f75 at zfsctl_root_lookup+0xf5
#12 0xffffffff82191e13 at zfsctl_umount_snapshots+0x83
#13 0xffffffff821aacfb at zfs_umount+0x7b
#14 0xffffffff80ad7c70 at dounmount+0x530
#15 0xffffffff80ae127b at vfs_unmountall+0x6b
#16 0xffffffff80ac24f9 at bufshutdown+0x3b9
#17 0xffffffff80a26fe9 at kern_reboot+0x189
lock order reversal:
 1st 0xfffff800125197c8 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:1222
 2nd 0xfffff8000ffbc240 devfs (devfs) @ /usr/src/sys/kern/vfs_subr.c:2498
stack backtrace:
#0 0xffffffff80a825a0 at witness_debugger+0x70
#1 0xffffffff80a824a1 at witness_checkorder+0xe71
#2 0xffffffff80a00a6b at __lockmgr_args+0xd3b
#3 0xffffffff80ace2bc at vop_stdlock+0x3c
#4 0xffffffff80fcb010 at VOP_LOCK1_APV+0x100
#5 0xffffffff80aeef9a at _vn_lock+0x9a
#6 0xffffffff80adf553 at vget+0x63
#7 0xffffffff808fcb4d at devfs_allocv+0xcd
#8 0xffffffff808fc653 at devfs_root+0x43
#9 0xffffffff80ad7b8f at dounmount+0x44f
#10 0xffffffff80ae12d4 at vfs_unmountall+0xc4
#11 0xffffffff80ac24f9 at bufshutdown+0x3b9
#12 0xffffffff80a26fe9 at kern_reboot+0x189
#13 0xffffffff80a26e03 at sys_reboot+0x3e3
#14 0xffffffff80e7a15b at amd64_syscall+0x2db
#15 0xffffffff80e5926b at Xfast_syscall+0xfb
Uptime: 7h20m6s


Fatal trap 9: general protection fault while in kernel mode
cpuid = 2; apic id = 02
instruction pointer	= 0x20:0xffffffff8215369b
stack pointer	        = 0x28:0xfffffe02329189b0
frame pointer	        = 0x28:0xfffffe02329189c0
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 0 (dbu_evict)
Uptime: 7h20m7s
Dumping 2338 out of 8050 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%

Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/zfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/zfs.ko
Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done.
done.
Loaded symbols for /boot/kernel/opensolaris.ko
Reading symbols from /boot/kernel/coretemp.ko...Reading symbols from /usr/lib/debug//boot/kernel/coretemp.ko.debug...done.
done.
Loaded symbols for /boot/kernel/coretemp.ko
Reading symbols from /boot/kernel/ichsmb.ko...Reading symbols from /usr/lib/debug//boot/kernel/ichsmb.ko.debug...done.
done.
Loaded symbols for /boot/kernel/ichsmb.ko
Reading symbols from /boot/kernel/smbus.ko...Reading symbols from /usr/lib/debug//boot/kernel/smbus.ko.debug...done.
done.
Loaded symbols for /boot/kernel/smbus.ko
Reading symbols from /boot/kernel/hwpmc.ko...Reading symbols from /usr/lib/debug//boot/kernel/hwpmc.ko.debug...done.
done.
Loaded symbols for /boot/kernel/hwpmc.ko
Reading symbols from /boot/kernel/iwn135fw.ko...Reading symbols from /usr/lib/debug//boot/kernel/iwn135fw.ko.debug...done.
done.
Loaded symbols for /boot/kernel/iwn135fw.ko
Reading symbols from /boot/kernel/aesni.ko...Reading symbols from /usr/lib/debug//boot/kernel/aesni.ko.debug...done.
done.
Loaded symbols for /boot/kernel/aesni.ko
Reading symbols from /boot/kernel/cryptodev.ko...Reading symbols from /usr/lib/debug//boot/kernel/cryptodev.ko.debug...done.
done.
Loaded symbols for /boot/kernel/cryptodev.ko
Reading symbols from /boot/kernel/dtraceall.ko...Reading symbols from /usr/lib/debug//boot/kernel/dtraceall.ko.debug...done.
done.
Loaded symbols for /boot/kernel/dtraceall.ko
Reading symbols from /boot/kernel/profile.ko...Reading symbols from /usr/lib/debug//boot/kernel/profile.ko.debug...done.
done.
Loaded symbols for /boot/kernel/profile.ko
Reading symbols from /boot/kernel/dtrace.ko...Reading symbols from /usr/lib/debug//boot/kernel/dtrace.ko.debug...done.
done.
Loaded symbols for /boot/kernel/dtrace.ko
Reading symbols from /boot/kernel/systrace_freebsd32.ko...Reading symbols from /usr/lib/debug//boot/kernel/systrace_freebsd32.ko.debug...done.
done.
Loaded symbols for /boot/kernel/systrace_freebsd32.ko
Reading symbols from /boot/kernel/systrace.ko...Reading symbols from /usr/lib/debug//boot/kernel/systrace.ko.debug...done.
done.
Loaded symbols for /boot/kernel/systrace.ko
Reading symbols from /boot/kernel/sdt.ko...Reading symbols from /usr/lib/debug//boot/kernel/sdt.ko.debug...done.
done.
Loaded symbols for /boot/kernel/sdt.ko
Reading symbols from /boot/kernel/fasttrap.ko...Reading symbols from /usr/lib/debug//boot/kernel/fasttrap.ko.debug...done.
done.
Loaded symbols for /boot/kernel/fasttrap.ko
Reading symbols from /boot/kernel/fbt.ko...Reading symbols from /usr/lib/debug//boot/kernel/fbt.ko.debug...done.
done.
Loaded symbols for /boot/kernel/fbt.ko
Reading symbols from /boot/kernel/dtnfscl.ko...Reading symbols from /usr/lib/debug//boot/kernel/dtnfscl.ko.debug...done.
done.
Loaded symbols for /boot/kernel/dtnfscl.ko
Reading symbols from /boot/kernel/dtmalloc.ko...Reading symbols from /usr/lib/debug//boot/kernel/dtmalloc.ko.debug...done.
done.
Loaded symbols for /boot/kernel/dtmalloc.ko
Reading symbols from /boot/kernel/cpuctl.ko...Reading symbols from /usr/lib/debug//boot/kernel/cpuctl.ko.debug...done.
done.
Loaded symbols for /boot/kernel/cpuctl.ko
Reading symbols from /boot/kernel/cuse.ko...Reading symbols from /usr/lib/debug//boot/kernel/cuse.ko.debug...done.
done.
Loaded symbols for /boot/kernel/cuse.ko
Reading symbols from /boot/kernel/ng_ubt.ko...Reading symbols from /usr/lib/debug//boot/kernel/ng_ubt.ko.debug...done.
done.
Loaded symbols for /boot/kernel/ng_ubt.ko
Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from /usr/lib/debug//boot/kernel/netgraph.ko.debug...done.
done.
Loaded symbols for /boot/kernel/netgraph.ko
Reading symbols from /boot/kernel/ng_hci.ko...Reading symbols from /usr/lib/debug//boot/kernel/ng_hci.ko.debug...done.
done.
Loaded symbols for /boot/kernel/ng_hci.ko
Reading symbols from /boot/kernel/ng_bluetooth.ko...Reading symbols from /usr/lib/debug//boot/kernel/ng_bluetooth.ko.debug...done.
done.
Loaded symbols for /boot/kernel/ng_bluetooth.ko
Reading symbols from /boot/kernel/ng_l2cap.ko...Reading symbols from /usr/lib/debug//boot/kernel/ng_l2cap.ko.debug...done.
done.
Loaded symbols for /boot/kernel/ng_l2cap.ko
Reading symbols from /boot/kernel/ng_btsocket.ko...Reading symbols from /usr/lib/debug//boot/kernel/ng_btsocket.ko.debug...done.
done.
Loaded symbols for /boot/kernel/ng_btsocket.ko
Reading symbols from /boot/kernel/ng_socket.ko...Reading symbols from /usr/lib/debug//boot/kernel/ng_socket.ko.debug...done.
done.
Loaded symbols for /boot/kernel/ng_socket.ko
#0  doadump (textdump=1) at pcpu.h:221
221	pcpu.h: No such file or directory.
	in pcpu.h
(kgdb) #0  doadump (textdump=1) at pcpu.h:221
#1  0xffffffff80a27235 in kern_reboot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:364
#2  0xffffffff80a2780b in vpanic (fmt=<value optimized out>, 
    ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:757
#3  0xffffffff80a27853 in panic (fmt=0x0)
    at /usr/src/sys/kern/kern_shutdown.c:688
#4  0xffffffff80383887 in db_panic (addr=<value optimized out>, 
    have_addr=false, count=0, modif=0x0) at /usr/src/sys/ddb/db_command.c:473
#5  0xffffffff80382ebe in db_command (cmd_table=0x0)
    at /usr/src/sys/ddb/db_command.c:440
#6  0xffffffff80382c54 in db_command_loop ()
    at /usr/src/sys/ddb/db_command.c:493
#7  0xffffffff803856eb in db_trap (type=<value optimized out>, code=0)
    at /usr/src/sys/ddb/db_main.c:251
#8  0xffffffff80a65773 in kdb_trap (type=9, code=0, tf=<value optimized out>)
    at /usr/src/sys/kern/subr_kdb.c:654
#9  0xffffffff80e79961 in trap_fatal (frame=0xfffffe0232918900, 
    eva=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:836
#10 0xffffffff80e7963a in trap (frame=<value optimized out>)
    at /usr/src/sys/amd64/amd64/trap.c:203
#11 0xffffffff80e58f87 in calltrap ()
    at /usr/src/sys/amd64/amd64/exception.S:234
#12 0xffffffff8215369b in txg_list_member (tl=0xfffff8001221c320, 
    p=<value optimized out>, txg=<value optimized out>)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c:850
#13 0xffffffff82112309 in dsl_dir_evict (dbu=0xfffff8000fedc400)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_dir.c:145
#14 0xffffffff80a75e30 in taskqueue_run_locked (queue=0xfffff80004557800)
    at /usr/src/sys/kern/subr_taskqueue.c:430
#15 0xffffffff80a76958 in taskqueue_thread_loop (arg=<value optimized out>)
    at /usr/src/sys/kern/subr_taskqueue.c:683
#16 0xffffffff809eb474 in fork_exit (
    callout=0xffffffff80a768d0 <taskqueue_thread_loop>, 
    arg=0xfffff80006b7c170, frame=0xfffffe0232918ac0)
    at /usr/src/sys/kern/kern_fork.c:1034
#17 0xffffffff80e594be in fork_trampoline ()
    at /usr/src/sys/amd64/amd64/exception.S:609
#18 0x0000000000000000 in ?? ()
Current language:  auto; currently minimal
(kgdb) 


Ideas?

Core IS available.


-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 214-642-9640                 E-Mail: ler at lerctr.org
US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961

More information about the freebsd-current mailing list