FreeBSD 10-STABLE/sparc64 panic

[Nathaniel W Filardo]

Well, I am having a bad day; I tried to roll back to 9 but ended up building
a 10 kernel again (too many source checkouts, sigh).  In any case, I kicked
WITNESS and INVARIANTS on this time and, while I cannot say for sure that
it is related, early in boot I am told

lock order reversal:
 1st 0xc0756998 entropy harvest mutex (entropy harvest mutex) @ /systank/src-git/sys/dev/random/random_harvestq.c:198
 2nd 0xfffff800055c7c38 uart_hwmtx (uart_hwmtx) @ /systank/src-git/sys/dev/uart/uart_cpu.h:94
KDB: stack backtrace:
_witness_debugger() at _witness_debugger+0x38
witness_checkorder() at witness_checkorder+0xea0
__mtx_lock_spin_flags() at __mtx_lock_spin_flags+0x134
uart_cnputc() at uart_cnputc+0x60
cnputc() at cnputc+0xac
putchar() at putchar+0xe8
kvprintf() at kvprintf+0x88
_vprintf() at _vprintf+0x38
vprintf() at vprintf+0x10
printf() at printf+0x20
witness_checkorder() at witness_checkorder+0xbf8
__mtx_lock_spin_flags() at __mtx_lock_spin_flags+0x134
sleepq_lock() at sleepq_lock+0x58
msleep_spin_sbt() at msleep_spin_sbt+0xb8
random_kthread() at random_kthread+0x294
fork_exit() at fork_exit+0xa4
fork_trampoline() at fork_trampoline+0x8
lock order reversal:
 1st 0xc0756998 entropy harvest mutex (entropy harvest mutex) @ /systank/src-git/sys/dev/random/random_harvestq.c:198
 2nd 0xc07b1510 sleepq chain (sleepq chain) @ /systank/src-git/sys/kern/subr_sleepqueue.c:237

The middle bit there looks like our favorite lockup:

spin lock 0xc07b3cb0 (smp rendezvous) held by 0xfffff80009116920 (tid 100335) too long
exclusive spin mutex smp rendezvous (smp rendezvous) r = 0 (0xc07b3cb0) locked @ /systank/src-git/sys/kern/subr_smp.c:497
timeout stopping cpus
panic: spin lock held too long
cpuid = 1
KDB: stack backtrace:
vpanic() at vpanic+0x1b4
panic() at panic+0x20
_mtx_lock_spin_failed() at _mtx_lock_spin_failed+0x74
_mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xb8
__mtx_lock_spin_flags() at __mtx_lock_spin_flags+0x190
tick_get_timecount_mp() at tick_get_timecount_mp+0x94
binuptime() at binuptime+0x3c
timercb() at timercb+0x6c
tick_intr() at tick_intr+0x220
-- interrupt level=0xe pil=0 %o7=0xc061d208 --
spinlock_exit() at spinlock_exit+0x2c
__mtx_unlock_spin_flags() at __mtx_unlock_spin_flags+0x138
uart_cnputc() at uart_cnputc+0xd0
cnputc() at cnputc+0xac
putchar() at putchar+0xe8
kvprintf() at kvprintf+0x890
_vprintf() at _vprintf+0x38
log() at log+0x48
do_link_state_change() at do_link_state_change+0x21c
taskqueue_run_locked() at taskqueue_run_locked+0x100
taskqueue_run() at taskqueue_run+0x64
taskqueue_swi_run() at taskqueue_swi_run+0x18
intr_event_execute_handlers() at intr_event_execute_handlers+0x154
ithread_loop() at ithread_loop+0x120
fork_exit() at fork_exit+0xa4
fork_trampoline() at fork_trampoline+0x8

So we might be seeing a deadlock between the console mutex and entropy
harvesting?

Cheers,
--nwf;
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-sparc64/attachments/20140520/a49f4ef9/attachment.sig>