Impact of changes made to umass.c at r232358

Tue Apr 17 23:54:46 UTC 2012

I am seeing a kernel panic in FreeBSD 8.1, which is reproduced after physically attaching and detaching a USB device several times.  The kernel debugger shows that the panic happens in camisr where the cam_sim and its associated mutex are clearly destroyed.  sim->refcount is 0, sim->softc is 1 (UMASS_GONE), and the sim->mtx is destroyed (mtx_lock = 6).

This looks very similar to FreeBSD PR kern/153514, which is unfortunately unresolved.

http://www.freebsd.org/cgi/query-pr.cgi?pr=153514

Is it possible that the changes made to umass.c at r232358 might fix this issue?

I currently have a machine in this state, so I can gather information from kdb if it will be helpful.  Here is some debug information that I have already collected:

db> show msgbuf
msgbufp = 0xffffffff84420fe0
magic = 63062, size = 65504, r= 53501, w = 54139, ptr = 0xffffffff84411000, cksum= 4373525
0:0): got CAM status 0xa
(da3:umass-sim0:0:0:0): fatal error, failed to attach to device
(da3:umass-sim0:0:0:0): removing device entry

Fatal trap 12: page fault while in kernel mode
cpuid = 3; apic id = 06
fault virtual address   = 0x290
fault code              = supervisor read data, page not present
instruction pointer     = 0x20:0xffffffff80284c71
stack pointer           = 0x28:0xffffff800014daf0
frame pointer           = 0x28:0xffffff800014db40
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 11 (swi2: cambio)
Kernel debug trap

Tracing pid 11 tid 100037 td 0xffffff0009014ba0
_mtx_lock_sleep() at _mtx_lock_sleep+0x71
_mtx_lock_flags() at _mtx_lock_flags+0xb8
camisr() at camisr+0xc6
intr_event_execute_handlers() at intr_event_execute_handlers+0x66
ithread_loop() at ithread_loop+0x8e
fork_exit() at fork_exit+0x112
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff800014dd30, rbp = 0 ---
db> show pcpu
cpuid        = 3
dynamic pcpu    = 0xffffff807fa22100
curthread    = 0xffffff0009014ba0: pid 11 "swi2: cambio"
curpcb       = 0xffffff800014dd40
fpcurthread  = none
idlethread   = 0xffffff0005f4f7c0: pid 10 "idle: cpu3"
curpmap         = 0
tssp            = 0xffffffff80848738
commontssp      = 0xffffffff80848738
rsp0            = 0xffffff800014dd40
gs32p           = 0xffffffff80847570
ldt             = 0xffffffff808475b0
tss             = 0xffffffff808475a0

db> show thread 100037
Thread 100037 at 0xffffff0009014ba0:
proc (pid 11): 0xffffff0005f48460
name: swi2: cambio
stack: 0xffffff800014a000-0xffffff800014dfff
flags: 0x10004  pflags: 0x210400
state: RUNNING (CPU 3)
priority: 44
container lock: sched lock 3 (0xffffffff8064f180)

db> show lock 0xffffffff8064f180
class: spin mutex
name: sched lock 3
flags: {SPIN, RECURSE}
state: {UNOWNED}

db> show registers
cs                0x20  WAKEUP_efer
ds                0x3b  WAKEUP_lstar+0x3
es            0x3b003b
fs          0x290001b0013
gs           0x290001b
ss                0x28  WAKEUP_pat
rax                0x6
rcx                  0
rdx                  0
rbx                0x4
rsp         0xffffff800014daf0
rbp         0xffffff800014db40
rsi         0xffffff0009014ba0
rdi         0xffffff017d0b5210
r8              0x1265  WAKEUP_cpu+0x1215
r9                   0
r10                  0
r11         0xffffffff80849ac8  __pcpu+0x7c8
r12         0xffffff017d0b5210
r13             0x1265  WAKEUP_cpu+0x1215
r14         0xffffff0009014ba0
r15                0x2
rip         0xffffffff80284c71  _mtx_lock_sleep+0x71
rflags         0x10246
_mtx_lock_sleep+0x71:   movl    0x290(%rcx),%ebx

db> show irqs
irq0: (no thread)
irq1: atkbd0 (pid 11)
irq3: uart1 (no thread)
irq4: uart0 (no thread)
irq5: (no thread)
irq6: (no thread)
irq7: (no thread)
irq8: (no thread)
irq9: acpi0 (pid 11)
irq10: (no thread)
irq11: (no thread)
irq12: (no thread)
irq13: (no thread)
irq14: (no thread)
irq15: (no thread)
irq16: (no thread)
irq17: (no thread)
irq18: (no thread)
irq19: (no thread)
irq20: atapci0 (pid 11) {ENTROPY}
irq21: (no thread)
irq22: ehci1 (pid 11)
irq23: ehci0 (pid 11)
irq32: (no thread)
irq33: (no thread)
irq34: (no thread)
irq35: (no thread)
irq36: (no thread)
irq37: (no thread)
irq38: (no thread)
irq39: (no thread)
irq40: (no thread)
irq41: (no thread)
irq42: (no thread)
irq43: (no thread)
irq44: (no thread)
irq45: (no thread)
irq46: (no thread)
irq47: (no thread)
irq48: (no thread)
irq49: (no thread)
irq50: (no thread)
irq51: (no thread)
irq52: (no thread)
irq53: (no thread)
irq54: (no thread)
irq55: (no thread)
irq64: (no thread)
irq65: (no thread)
irq66: (no thread)
irq67: (no thread)
irq68: (no thread)
irq69: (no thread)
irq70: (no thread)
irq71: (no thread)
irq72: (no thread)
irq73: (no thread)
irq74: (no thread)
irq75: (no thread)
irq76: (no thread)
irq77: (no thread)
irq78: (no thread)
irq79: (no thread)
irq80: (no thread)
irq81: (no thread)
irq82: (no thread)
irq83: (no thread)
irq84: (no thread)
irq85: (no thread)
irq86: (no thread)
irq87: (no thread)
irq256: ix0:que 0 (pid 11)
irq257: ix0:que 1 (pid 11)
irq258: ix0:link (pid 11)
irq259: ix1:que 0 (pid 11)
irq260: ix1:que 1 (pid 11)
irq261: ix1:link (pid 11)
irq262: cmlpci0 (pid 11)
irq263: cmlpci1 (pid 11)
irq264: cmlpci2 (pid 11)
irq265: cmlpci3 (pid 11)
irq266: igb0:que 0 (pid 11)
irq267: igb0:que 1 (pid 11)
irq268: igb0:que 2 (pid 11)
irq269: igb0:que 3 (pid 11)
irq270: igb0:link (pid 11)
irq271: igb1:que 0 (pid 11)
irq272: igb1:que 1 (pid 11)
irq273: igb1:que 2 (pid 11)
irq274: igb1:que 3 (pid 11)
irq275: igb1:link (pid 11)

Terence Telkamp
Storage Development Associate Engineer II
Dell | Compellent