[Bug 281560] gve (4) uma deadlock during high tcp throughput
Date: Tue, 17 Sep 2024 21:21:46 UTC
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=281560
Bug ID: 281560
Summary: gve (4) uma deadlock during high tcp throughput
Product: Base System
Version: 14.1-RELEASE
Hardware: amd64
OS: Any
Status: New
Severity: Affects Only Me
Priority: ---
Component: kern
Assignee: bugs@FreeBSD.org
Reporter: shailend@google.com
In https://reviews.freebsd.org/D46690 I am adding support for a newer queue
format to gve that supports ~200Gbps of throughput.
While stress testing these changes I ran into a deadlock that I'm unable to
find a root cause for. This is reproduced by having two VMs sending
bidirectional tcp traffic to each other, and the deadlock sets in after a few
hours.
I am listing the stack traces of the driver taskqueues after the deadlock sets
in for two different occurrences of the deadlock:
```
[root@FreeBSD14 ~]# for locktid in $(procstat -ta | grep gve | grep lock | cut
-d" " -f6); do procstat -k $locktid | grep $locktid; done
0 100416 kernel gve0 rxq 0 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_m_copym tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_rx_done
0 100417 kernel gve0 rxq 1 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_m_copym tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_rx_done
0 100418 kernel gve0 rxq 2 mi_switch __rw_wlock_hard
_rw_wlock_cookie _inp_smr_lock in_pcblookup_hash_smr in_pcblookup_mbuf
tcp_input_with_port tcp_input ip_input netisr_dispatch_src ether_demux
ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush tcp_lro_rx_done
tcp_lro_flush_all gve_rx_cleanup_tq_dqo
0 100419 kernel gve0 rxq 3 mi_switch __rw_wlock_hard
_rw_wlock_cookie _inp_smr_lock in_pcblookup_hash_smr in_pcblookup_mbuf
tcp_input_with_port tcp_input ip_input netisr_dispatch_src ether_demux
ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush tcp_lro_rx_done
tcp_lro_flush_all gve_rx_cleanup_tq_dqo
0 100420 kernel gve0 rxq 4 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_m_copym tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_rx_done
0 100421 kernel gve0 rxq 5 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_m_copym tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_flush_active
0 100422 kernel gve0 rxq 6 mi_switch __rw_wlock_hard
_rw_wlock_cookie _inp_smr_lock in_pcblookup_hash_smr in_pcblookup_mbuf
tcp_input_with_port tcp_input ip_input netisr_dispatch_src ether_demux
ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush tcp_lro_rx_done
tcp_lro_flush_all gve_rx_cleanup_tq_dqo
0 100423 kernel gve0 rxq 7 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_m_copym tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_rx_done
0 100424 kernel gve0 rxq 8 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_m_copym tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_rx_done
0 100425 kernel gve0 rxq 9 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_m_copym tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_flush_active
0 100426 kernel gve0 rxq 10 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_m_copym tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_rx_done
0 100427 kernel gve0 rxq 11 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_m_copym tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_rx_done
0 100428 kernel gve0 rxq 12 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_m_copym tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_rx_done
0 100429 kernel gve0 rxq 13 mi_switch __rw_wlock_hard
_rw_wlock_cookie _inp_smr_lock in_pcblookup_hash_smr in_pcblookup_mbuf
tcp_input_with_port tcp_input ip_input netisr_dispatch_src ether_demux
ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush tcp_lro_rx_done
tcp_lro_flush_all gve_rx_cleanup_tq_dqo
0 100430 kernel gve0 rxq 14 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_rx_done tcp_lro_flush_all
0 100431 kernel gve0 rxq 15 mi_switch __rw_wlock_hard
_rw_wlock_cookie _inp_smr_lock in_pcblookup_hash_smr in_pcblookup_mbuf
tcp_input_with_port tcp_input ip_input netisr_dispatch_src ether_demux
ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush tcp_lro_rx_done
tcp_lro_flush_all gve_rx_cleanup_tq_dqo
0 100434 kernel gve0 txq 1 mi_switch __mtx_lock_sleep
__mtx_lock_flags zone_put_bucket cache_free uma_zfree_arg m_free m_freem
gve_handle_packet_completion gve_tx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit fork_trampoline
0 100448 kernel gve0 txq 8 mi_switch __mtx_lock_sleep
__mtx_lock_flags zone_put_bucket cache_free uma_zfree_arg m_free m_freem
gve_handle_packet_completion gve_tx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit fork_trampoline
0 100456 kernel gve0 txq 12 mi_switch __mtx_lock_sleep
__mtx_lock_flags zone_put_bucket cache_free uma_zfree_arg m_free m_freem
gve_handle_packet_completion gve_tx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit fork_trampoline
0 100458 kernel gve0 txq 13 mi_switch __mtx_lock_sleep
__mtx_lock_flags zone_put_bucket cache_free uma_zfree_arg m_free m_freem
gve_handle_packet_completion gve_tx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit fork_trampoline
```
```
[root@FreeBSD14 ~]# for locktid in $(procstat -ta | grep gve | grep lock | cut
-d" " -f6); do procstat -k $locktid | grep $locktid; done
0 100433 kernel gve0 rxq 0 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_default_output tcp_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_flush_all
0 100434 kernel gve0 rxq 1 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_default_output tcp_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_flush_all
0 100435 kernel gve0 rxq 2 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_default_output tcp_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_flush_all
0 100436 kernel gve0 rxq 3 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_flush_all gve_rx_cleanup_tq_dqo
0 100437 kernel gve0 rxq 4 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_flush_all gve_rx_cleanup_tq_dqo
0 100438 kernel gve0 rxq 5 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_flush_all gve_rx_cleanup_tq_dqo
0 100439 kernel gve0 rxq 6 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_m_copym tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_flush_all
0 100440 kernel gve0 rxq 7 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_m_copym tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_flush_all
0 100441 kernel gve0 rxq 8 mi_switch __rw_wlock_hard
_rw_wlock_cookie _inp_smr_lock in_pcblookup_hash_smr in_pcblookup_mbuf
tcp_input_with_port tcp_input ip_input netisr_dispatch_src ether_demux
ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush tcp_lro_flush_all
gve_rx_cleanup_tq_dqo taskqueue_run_locked
0 100442 kernel gve0 rxq 9 mi_switch __rw_wlock_hard
_rw_wlock_cookie _inp_smr_lock in_pcblookup_hash_smr in_pcblookup_mbuf
tcp_input_with_port tcp_input ip_input netisr_dispatch_src ether_demux
ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush tcp_lro_flush_all
gve_rx_cleanup_tq_dqo taskqueue_run_locked
0 100443 kernel gve0 rxq 10 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_m_copym tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_flush_all
0 100444 kernel gve0 rxq 11 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_m_copym tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_flush_all
0 100445 kernel gve0 rxq 12 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_m_copym tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_flush_all
0 100446 kernel gve0 rxq 13 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_m_copym tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_flush_all
0 100447 kernel gve0 rxq 14 mi_switch __rw_wlock_hard
_rw_wlock_cookie _inp_smr_lock in_pcblookup_hash_smr in_pcblookup_mbuf
tcp_input_with_port tcp_input ip_input netisr_dispatch_src ether_demux
ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush tcp_lro_flush_all
gve_rx_cleanup_tq_dqo taskqueue_run_locked
0 100448 kernel gve0 rxq 15 mi_switch __mtx_lock_sleep
__mtx_lock_flags cache_alloc cache_alloc_retry tcp_m_copym tcp_default_output
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
tcp_lro_flush_all
0 100457 kernel gve0 txq 4 mi_switch __mtx_lock_sleep
__mtx_lock_flags zone_put_bucket cache_free uma_zfree_arg m_free m_freem
gve_handle_packet_completion gve_tx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit fork_trampoline
0 100463 kernel gve0 txq 7 mi_switch __mtx_lock_sleep
__mtx_lock_flags zone_put_bucket cache_free uma_zfree_arg m_free m_freem
gve_handle_packet_completion gve_tx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit fork_trampoline
0 100465 kernel gve0 txq 8 mi_switch __mtx_lock_sleep
__mtx_lock_flags zone_put_bucket cache_free uma_zfree_arg m_free m_freem
gve_handle_packet_completion gve_tx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit fork_trampoline
0 100467 kernel gve0 txq 9 mi_switch __mtx_lock_sleep
__mtx_lock_flags zone_put_bucket cache_free uma_zfree_arg m_free m_freem
gve_handle_packet_completion gve_tx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit fork_trampoline
0 100469 kernel gve0 txq 10 mi_switch __mtx_lock_sleep
__mtx_lock_flags zone_put_bucket cache_free uma_zfree_arg mb_free_ext m_free
m_freem gve_handle_packet_completion gve_tx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit fork_trampoline
```
I tried to look into any incorrect kpi use in the driver
(https://reviews.freebsd.org/D46690) but couldn't find any offenses. It is
rather straightforward: in the Rx paths it inputs cluster mbufs and m_freems
transmitted mbufs in the Tx path. Thus I'm hesitantly suspecting some race
condition in the uma stack given the traces.
I reproduced the issue with INVARIANTS and WITNESS but they didn't catch
anything.
The issue also reproduces with a slightly different datapath mode in the driver
(https://reviews.freebsd.org/D46691). In this mode, it is not cluster mbufs
that are input in the Rx path usually, but rather mbufs with external storage
attached. The stack traces of the driver taskqueues after the deadlock look
like so:
```
[root@FreeBSD13 ~]# for locktid in $(procstat -ta | grep gve | grep lock | cut
-d" " -f6); do procstat -k $locktid | grep $locktid; done
0 100451 kernel gve0 rxq 1 mi_switch __rw_wlock_hard
in_pcblookup_hash in_pcblookup_mbuf tcp_input_with_port tcp_input ip_input
netisr_dispatch_src ether_demux ether_nh_input netisr_dispatch_src ether_input
tcp_lro_flush tcp_lro_flush_all gve_rx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit
0 100452 kernel gve0 rxq 2 mi_switch __mtx_lock_sleep
cache_alloc cache_alloc_retry m_get2 gve_rx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit fork_trampoline
0 100453 kernel gve0 rxq 3 mi_switch __mtx_lock_sleep
cache_alloc cache_alloc_retry m_get2 gve_rx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit fork_trampoline
0 100454 kernel gve0 rxq 4 mi_switch __rw_wlock_hard
in_pcblookup_hash in_pcblookup_mbuf tcp_input_with_port tcp_input ip_input
netisr_dispatch_src ether_demux ether_nh_input netisr_dispatch_src ether_input
tcp_lro_flush tcp_lro_flush_all gve_rx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit
0 100455 kernel gve0 rxq 5 mi_switch __rw_wlock_hard
in_pcblookup_hash in_pcblookup_mbuf tcp_input_with_port tcp_input ip_input
netisr_dispatch_src ether_demux ether_nh_input netisr_dispatch_src ether_input
tcp_lro_flush tcp_lro_flush_all gve_rx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit
0 100457 kernel gve0 rxq 7 mi_switch __rw_wlock_hard
in_pcblookup_hash in_pcblookup_mbuf tcp_input_with_port tcp_input ip_input
netisr_dispatch_src ether_demux ether_nh_input netisr_dispatch_src ether_input
tcp_lro_flush tcp_lro_flush_all gve_rx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit
0 100458 kernel gve0 rxq 8 mi_switch __rw_wlock_hard
in_pcblookup_hash in_pcblookup_mbuf tcp_input_with_port tcp_input ip_input
netisr_dispatch_src ether_demux ether_nh_input netisr_dispatch_src ether_input
tcp_lro_flush tcp_lro_flush_all gve_rx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit
0 100459 kernel gve0 rxq 9 mi_switch __mtx_lock_sleep
zone_put_bucket cache_free uma_zfree_arg mb_free_ext m_free m_freem
tcp_do_segment tcp_input_with_port tcp_input ip_input netisr_dispatch_src
ether_demux ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush
0 100460 kernel gve0 rxq 10 mi_switch __rw_wlock_hard
in_pcblookup_hash in_pcblookup_mbuf tcp_input_with_port tcp_input ip_input
netisr_dispatch_src ether_demux ether_nh_input netisr_dispatch_src ether_input
tcp_lro_flush tcp_lro_flush_all gve_rx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit
0 100461 kernel gve0 rxq 11 mi_switch __mtx_lock_sleep
cache_alloc cache_alloc_retry m_get2 gve_rx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit fork_trampoline
0 100462 kernel gve0 rxq 12 mi_switch __mtx_lock_sleep
cache_alloc cache_alloc_retry tcp_m_copym tcp_output tcp_do_segment
tcp_input_with_port tcp_input ip_input netisr_dispatch_src ether_demux
ether_nh_input netisr_dispatch_src ether_input tcp_lro_flush tcp_lro_flush_all
gve_rx_cleanup_tq_dqo
0 100463 kernel gve0 rxq 13 mi_switch __mtx_lock_sleep
cache_alloc cache_alloc_retry m_get2 gve_rx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit fork_trampoline
0 100464 kernel gve0 rxq 14 mi_switch __rw_wlock_hard
in_pcblookup_hash in_pcblookup_mbuf tcp_input_with_port tcp_input ip_input
netisr_dispatch_src ether_demux ether_nh_input netisr_dispatch_src ether_input
tcp_lro_flush tcp_lro_flush_all gve_rx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit
0 100465 kernel gve0 rxq 15 mi_switch __mtx_lock_sleep
cache_alloc cache_alloc_retry m_get2 gve_rx_cleanup_tq_dqo taskqueue_run_locked
taskqueue_thread_loop fork_exit fork_trampoline
0 100466 kernel gve0 txq 0 mi_switch __mtx_lock_sleep
zone_put_bucket cache_free uma_zfree_arg m_free m_freem gve_tx_cleanup_tq_dqo
taskqueue_run_locked taskqueue_thread_loop fork_exit fork_trampoline
0 100468 kernel gve0 txq 1 mi_switch __mtx_lock_sleep
zone_put_bucket cache_free uma_zfree_arg m_free m_freem gve_tx_cleanup_tq_dqo
taskqueue_run_locked taskqueue_thread_loop fork_exit fork_trampoline
0 100470 kernel gve0 txq 2 mi_switch __mtx_lock_sleep
zone_put_bucket cache_free uma_zfree_arg m_free m_freem gve_tx_cleanup_tq_dqo
taskqueue_run_locked taskqueue_thread_loop fork_exit fork_trampoline
0 100472 kernel gve0 txq 3 mi_switch __mtx_lock_sleep
zone_put_bucket cache_free uma_zfree_arg m_free m_freem gve_tx_cleanup_tq_dqo
taskqueue_run_locked taskqueue_thread_loop fork_exit fork_trampoline
0 100474 kernel gve0 txq 4 mi_switch __mtx_lock_sleep
zone_put_bucket cache_free uma_zfree_arg m_free m_freem gve_tx_cleanup_tq_dqo
taskqueue_run_locked taskqueue_thread_loop fork_exit fork_trampoline
0 100480 kernel gve0 txq 7 mi_switch __mtx_lock_sleep
zone_put_bucket cache_free uma_zfree_arg m_free m_freem gve_tx_cleanup_tq_dqo
taskqueue_run_locked taskqueue_thread_loop fork_exit fork_trampoline
0 100482 kernel gve0 txq 8 mi_switch __mtx_lock_sleep
zone_put_bucket cache_free uma_zfree_arg m_free m_freem gve_tx_cleanup_tq_dqo
taskqueue_run_locked taskqueue_thread_loop fork_exit fork_trampoline
0 100484 kernel gve0 txq 9 mi_switch __mtx_lock_sleep
zone_put_bucket cache_free uma_zfree_arg m_free m_freem gve_tx_cleanup_tq_dqo
taskqueue_run_locked taskqueue_thread_loop fork_exit fork_trampoline
0 100486 kernel gve0 txq 10 mi_switch __mtx_lock_sleep
zone_put_bucket cache_free uma_zfree_arg m_free m_freem gve_tx_cleanup_tq_dqo
taskqueue_run_locked taskqueue_thread_loop fork_exit fork_trampoline
0 100488 kernel gve0 txq 11 mi_switch __mtx_lock_sleep
zone_put_bucket cache_free uma_zfree_arg m_free m_freem gve_tx_cleanup_tq_dqo
taskqueue_run_locked taskqueue_thread_loop fork_exit fork_trampoline
0 100490 kernel gve0 txq 12 mi_switch __mtx_lock_sleep
zone_put_bucket cache_free uma_zfree_arg m_free m_freem gve_tx_cleanup_tq_dqo
taskqueue_run_locked taskqueue_thread_loop fork_exit fork_trampoline
0 100492 kernel gve0 txq 13 mi_switch __mtx_lock_sleep
zone_put_bucket cache_free uma_zfree_arg m_free m_freem gve_tx_cleanup_tq_dqo
taskqueue_run_locked taskqueue_thread_loop fork_exit fork_trampoline
0 100494 kernel gve0 txq 14 mi_switch __mtx_lock_sleep
zone_put_bucket cache_free uma_zfree_arg m_free m_freem gve_tx_cleanup_tq_dqo
taskqueue_run_locked taskqueue_thread_loop fork_exit fork_trampoline
0 100496 kernel gve0 txq 15 mi_switch __mtx_lock_sleep
zone_put_bucket cache_free uma_zfree_arg m_free m_freem gve_tx_cleanup_tq_dqo
taskqueue_run_locked taskqueue_thread_loop fork_exit fork_trampoline
```
I've reproduced this in 14-1 and 13-1.
Any guidance or help in uncovering the source of this deadlock would be very
helpful!
--
You are receiving this mail because:
You are the assignee for the bug.