[ZFS][PANIC] Solaris Assert/zio.c:2548

Larry Rosenman ler at lerctr.org
Sun Jul 20 19:20:48 UTC 2014 

On 2014-07-20 14:18, Steven Hartland wrote:
> Can you provide the details of the zio which caused the panic?
> 
> Also does any of your pools support trim?

No, on the trim.  Can you walk me through getting the zio you need?

> 
>    Regards
>    Steve
> 
> ----- Original Message ----- From: "Larry Rosenman" <ler at lerctr.org>
> To: <freebsd-fs at freebsd.org>; <freebsd-current at freebsd.org>
> Sent: Sunday, July 20, 2014 3:03 PM
> Subject: [ZFS][PANIC] Solaris Assert/zio.c:2548
> 
> 
>> Got the following panic overnight (I think while a nightly rsync was 
>> running):
>> 
>> Dump header from device /dev/gpt/swap0
>>  Architecture: amd64
>>  Architecture Version: 2
>>  Dump Length: 8122101760B (7745 MB)
>>  Blocksize: 512
>>  Dumptime: Sun Jul 20 03:22:18 2014
>>  Hostname: borg.lerctr.org
>>  Magic: FreeBSD Kernel Dump
>>  Version String: FreeBSD 11.0-CURRENT #50 r268894M: Sat Jul 19 
>> 18:06:08 CDT 2014
>>    root at borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER
>>  Panic String: solaris assert: !(zio->io_flags & ZIO_FLAG_DELEGATED), 
>> file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, 
>> line: 2874
>>  Dump Parity: 763150733
>>  Bounds: 5
>>  Dump Status: good
>> 
>> 
>> borg.lerctr.org dumped core - see /var/crash/vmcore.5
>> 
>> Sun Jul 20 03:28:12 CDT 2014
>> 
>> FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #50 
>> r268894M: Sat Jul 19 18:06:08 CDT 2014 
>> root at borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER  amd64
>> 
>> panic: solaris assert: !(zio->io_flags & ZIO_FLAG_DELEGATED), file: 
>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 
>> 2874
>> 
>> GNU gdb 6.1.1 [FreeBSD]
>> Copyright 2004 Free Software Foundation, Inc.
>> GDB is free software, covered by the GNU General Public License, and 
>> you are
>> welcome to change it and/or distribute copies of it under certain 
>> conditions.
>> Type "show copying" to see the conditions.
>> There is absolutely no warranty for GDB.  Type "show warranty" for 
>> details.
>> This GDB was configured as "amd64-marcel-freebsd"...
>> 
>> Unread portion of the kernel message buffer:
>> panic: solaris assert: !(zio->io_flags & ZIO_FLAG_DELEGATED), file: 
>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 
>> 2874
>> cpuid = 7
>> KDB: stack backtrace:
>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 
>> 0xfffffe100c49f930
>> kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe100c49f9e0
>> vpanic() at vpanic+0x126/frame 0xfffffe100c49fa20
>> panic() at panic+0x43/frame 0xfffffe100c49fa80
>> assfail() at assfail+0x1d/frame 0xfffffe100c49fa90
>> zio_vdev_io_assess() at zio_vdev_io_assess+0x2ed/frame 
>> 0xfffffe100c49fac0
>> zio_execute() at zio_execute+0x1e9/frame 0xfffffe100c49fb20
>> taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 
>> 0xfffffe100c49fb80
>> taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 
>> 0xfffffe100c49fbb0
>> fork_exit() at fork_exit+0x84/frame 0xfffffe100c49fbf0
>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe100c49fbf0
>> --- trap 0, rip = 0, rsp = 0xfffffe100c49fcb0, rbp = 0 ---
>> Uptime: 8h57m17s
>> (ada2:ahcich2:0:0:0): FLUSHCACHE48. ACB: ea 00 00 00 00 40 00 00 00 00 
>> 00 00
>> (ada2:ahcich2:0:0:0): CAM status: Command timeout
>> (ada2:ahcich2:0:0:0): Error 5, Retries exhausted
>> (ada2:ahcich2:0:0:0): Synchronize cache failed
>> Dumping 7745 out of 64463 
>> MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
>> 
>> Reading symbols from /boot/kernel/linux.ko.symbols...done.
>> Loaded symbols for /boot/kernel/linux.ko.symbols
>> Reading symbols from /boot/kernel/if_lagg.ko.symbols...done.
>> Loaded symbols for /boot/kernel/if_lagg.ko.symbols
>> Reading symbols from /boot/kernel/snd_envy24ht.ko.symbols...done.
>> Loaded symbols for /boot/kernel/snd_envy24ht.ko.symbols
>> Reading symbols from /boot/kernel/snd_spicds.ko.symbols...done.
>> Loaded symbols for /boot/kernel/snd_spicds.ko.symbols
>> Reading symbols from /boot/kernel/coretemp.ko.symbols...done.
>> Loaded symbols for /boot/kernel/coretemp.ko.symbols
>> Reading symbols from /boot/kernel/ichsmb.ko.symbols...done.
>> Loaded symbols for /boot/kernel/ichsmb.ko.symbols
>> Reading symbols from /boot/kernel/smbus.ko.symbols...done.
>> Loaded symbols for /boot/kernel/smbus.ko.symbols
>> Reading symbols from /boot/kernel/ichwd.ko.symbols...done.
>> Loaded symbols for /boot/kernel/ichwd.ko.symbols
>> Reading symbols from /boot/kernel/cpuctl.ko.symbols...done.
>> Loaded symbols for /boot/kernel/cpuctl.ko.symbols
>> Reading symbols from /boot/kernel/crypto.ko.symbols...done.
>> Loaded symbols for /boot/kernel/crypto.ko.symbols
>> Reading symbols from /boot/kernel/cryptodev.ko.symbols...done.
>> Loaded symbols for /boot/kernel/cryptodev.ko.symbols
>> Reading symbols from /boot/kernel/dtraceall.ko.symbols...done.
>> Loaded symbols for /boot/kernel/dtraceall.ko.symbols
>> Reading symbols from /boot/kernel/profile.ko.symbols...done.
>> Loaded symbols for /boot/kernel/profile.ko.symbols
>> Reading symbols from /boot/kernel/cyclic.ko.symbols...done.
>> Loaded symbols for /boot/kernel/cyclic.ko.symbols
>> Reading symbols from /boot/kernel/dtrace.ko.symbols...done.
>> Loaded symbols for /boot/kernel/dtrace.ko.symbols
>> Reading symbols from 
>> /boot/kernel/systrace_freebsd32.ko.symbols...done.
>> Loaded symbols for /boot/kernel/systrace_freebsd32.ko.symbols
>> Reading symbols from /boot/kernel/systrace.ko.symbols...done.
>> Loaded symbols for /boot/kernel/systrace.ko.symbols
>> Reading symbols from /boot/kernel/sdt.ko.symbols...done.
>> Loaded symbols for /boot/kernel/sdt.ko.symbols
>> Reading symbols from /boot/kernel/lockstat.ko.symbols...done.
>> Loaded symbols for /boot/kernel/lockstat.ko.symbols
>> Reading symbols from /boot/kernel/fasttrap.ko.symbols...done.
>> Loaded symbols for /boot/kernel/fasttrap.ko.symbols
>> Reading symbols from /boot/kernel/fbt.ko.symbols...done.
>> Loaded symbols for /boot/kernel/fbt.ko.symbols
>> Reading symbols from /boot/kernel/dtnfscl.ko.symbols...done.
>> Loaded symbols for /boot/kernel/dtnfscl.ko.symbols
>> Reading symbols from /boot/kernel/dtmalloc.ko.symbols...done.
>> Loaded symbols for /boot/kernel/dtmalloc.ko.symbols
>> Reading symbols from /boot/modules/vboxdrv.ko...done.
>> Loaded symbols for /boot/modules/vboxdrv.ko
>> Reading symbols from /boot/kernel/ipmi.ko.symbols...done.
>> Loaded symbols for /boot/kernel/ipmi.ko.symbols
>> Reading symbols from /boot/kernel/ipmi_linux.ko.symbols...done.
>> Loaded symbols for /boot/kernel/ipmi_linux.ko.symbols
>> Reading symbols from /boot/kernel/radeonkms.ko.symbols...done.
>> Loaded symbols for /boot/kernel/radeonkms.ko.symbols
>> Reading symbols from /boot/kernel/iicbb.ko.symbols...done.
>> Loaded symbols for /boot/kernel/iicbb.ko.symbols
>> Reading symbols from /boot/kernel/iicbus.ko.symbols...done.
>> Loaded symbols for /boot/kernel/iicbus.ko.symbols
>> Reading symbols from /boot/kernel/iic.ko.symbols...done.
>> Loaded symbols for /boot/kernel/iic.ko.symbols
>> Reading symbols from /boot/kernel/drm2.ko.symbols...done.
>> Loaded symbols for /boot/kernel/drm2.ko.symbols
>> Reading symbols from 
>> /boot/kernel/radeonkmsfw_R100_cp.ko.symbols...done.
>> Loaded symbols for /boot/kernel/radeonkmsfw_R100_cp.ko.symbols
>> Reading symbols from /boot/kernel/fdescfs.ko.symbols...done.
>> Loaded symbols for /boot/kernel/fdescfs.ko.symbols
>> Reading symbols from /boot/kernel/linprocfs.ko.symbols...done.
>> Loaded symbols for /boot/kernel/linprocfs.ko.symbols
>> Reading symbols from /boot/kernel/uhid.ko.symbols...done.
>> Loaded symbols for /boot/kernel/uhid.ko.symbols
>> Reading symbols from /boot/modules/vboxnetflt.ko...done.
>> Loaded symbols for /boot/modules/vboxnetflt.ko
>> Reading symbols from /boot/kernel/netgraph.ko.symbols...done.
>> Loaded symbols for /boot/kernel/netgraph.ko.symbols
>> Reading symbols from /boot/kernel/ng_ether.ko.symbols...done.
>> Loaded symbols for /boot/kernel/ng_ether.ko.symbols
>> Reading symbols from /boot/modules/vboxnetadp.ko...done.
>> Loaded symbols for /boot/modules/vboxnetadp.ko
>> #0  doadump (textdump=1) at pcpu.h:219
>> 219 pcpu.h: No such file or directory.
>> in pcpu.h
>> (kgdb) #0  doadump (textdump=1) at pcpu.h:219
>> #1  0xffffffff80a055f7 in kern_reboot (howto=260)
>>    at /usr/src/sys/kern/kern_shutdown.c:445
>> #2  0xffffffff80a05b35 in vpanic (fmt=<value optimized out>,
>>    ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:744
>> #3  0xffffffff80a05b83 in panic (fmt=0x0)
>>    at /usr/src/sys/kern/kern_shutdown.c:673
>> #4  0xffffffff8032d05d in assfail (a=<value optimized out>,
>>    f=<value optimized out>, l=<value optimized out>)
>>    at 
>> /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:81
>> #5  0xffffffff8040ad6d in zio_vdev_io_assess (ziop=<value optimized 
>> out>)
>>    at 
>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2874
>> #6  0xffffffff80405dd9 in zio_execute (zio=0xfffff809e449b730)
>>    at 
>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1416
>> #7  0xffffffff80a4de60 in taskqueue_run_locked 
>> (queue=0xfffff8002255e800)
>>    at /usr/src/sys/kern/subr_taskqueue.c:356
>> #8  0xffffffff80a4e8db in taskqueue_thread_loop (arg=<value optimized 
>> out>)
>>    at /usr/src/sys/kern/subr_taskqueue.c:623
>> #9  0xffffffff809d3cc4 in fork_exit (
>>    callout=0xffffffff80a4e840 <taskqueue_thread_loop>,
>>    arg=0xfffff80022611470, frame=0xfffffe100c49fc00)
>>    at /usr/src/sys/kern/kern_fork.c:977
>> #10 0xffffffff80df5afe in fork_trampoline ()
>>    at /usr/src/sys/amd64/amd64/exception.S:605
>> #11 0x0000000000000000 in ?? ()
>> Current language:  auto; currently minimal
>> (kgdb)
>> 
>> 
>> vmcore is available.
>> 
>> I re-ran the rsync, and no panic.
>> 
>> Ideas?
>> 
>> 
>> -- Larry Rosenman                     http://www.lerctr.org/~ler
>> Phone: +1 214-642-9640                 E-Mail: ler at lerctr.org
>> US Mail: 108 Turvey Cove, Hutto, TX 78634-5688
>> _______________________________________________
>> freebsd-current at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-current
>> To unsubscribe, send any mail to 
>> "freebsd-current-unsubscribe at freebsd.org"
>> 
-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 214-642-9640 (c)     E-Mail: ler at lerctr.org
US Mail: 108 Turvey Cove, Hutto, TX 78634-5688

More information about the freebsd-current mailing list