ixgbe rx & tx locks

[Ryan Stone]

On Wed, Sep 26, 2012 at 3:38 PM, John Baldwin <jhb at freebsd.org> wrote:
>> ix:rx -> udp -> udpinp -> in_multi_mtx -> ix:core -> ix:rx
>
> Hmm, I'm not sure where the 'in_multi_mtx -> ix:core' bit comes from.
> I think that is the broken part of this.  The SIOCADDMULTI and SIOCDELMULTI
> ioctls are invoked without any stack locks held, so it shouldn't come from
> there.

I gave the backtrace for this part.  in_joingroup_locked is called
with the in_multi_mtx held, which calls if_addmulti, which calls the
ioctl handler.

lock order reversal:
 1st 0xffffffff80ae2440 in_multi_mtx (in_multi_mtx) @
src/sys/netinet/in_mcast.c:1095
 2nd 0xffffff8001539400 ixgbe0 (IXGBE Core Lock) @
src/sys/dev/ixgbe/ixgbe.c:1725
KDB: stack backtrace:
db_trace_self_wrapper() at 0xffffffff801dd5aa = db_trace_self_wrapper+0x2a
_witness_debugger() at 0xffffffff8044411e = _witness_debugger+0x2e
witness_checkorder() at 0xffffffff804453c7 = witness_checkorder+0x807
_mtx_lock_flags() at 0xffffffff803ec3ba = _mtx_lock_flags+0x8a
ixgbe_ioctl() at 0xffffffff802e07ae = ixgbe_ioctl+0x60e
if_addmulti() at 0xffffffff804a9f7b = if_addmulti+0x19b
in_joingroup_locked() at 0xffffffff804db8ec = in_joingroup_locked+0x1bc
in_joingroup() at 0xffffffff804dd5a2 = in_joingroup+0x52
in_control() at 0xffffffff804d7a70 = in_control+0x1160
ifioctl() at 0xffffffff804adec6 = ifioctl+0x5b6
nfs_mountroot() at 0xffffffff80567244 = nfs_mountroot+0x94
nfs_mount() at 0xffffffff80567b7b = nfs_mount+0x4db
vfs_donmount() at 0xffffffff8048919e = vfs_donmount+0xcde
kernel_mount() at 0xffffffff80489a71 = kernel_mount+0xa1
vfs_mountroot_try() at 0xffffffff80489f9d = vfs_mountroot_try+0x17d
vfs_mountroot() at 0xffffffff8048ab7d = vfs_mountroot+0x4fd
start_init() at 0xffffffff803b2932 = start_init+0x62
fork_exit() at 0xffffffff803d1fba = fork_exit+0x12a
fork_trampoline() at 0xffffffff805f582e = fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000042d00, rbp = 0 ---