[Differential] D20824: Fix netmap + vlan panics
aleksandr.fedorov_itglobal.com (Aleksandr Fedorov)
phabric-noreply at FreeBSD.org
Mon Jul 1 12:58:56 UTC 2019
aleksandr.fedorov_itglobal.com created this revision.
aleksandr.fedorov_itglobal.com added reviewers: vmaffione, jhb, bz.
REVISION SUMMARY
It is very useful to be able to create virtual networks, such as: VM1 - VALE switch - if_vlan - ix0 -- physical network -- ix0 - if_vlan - VALE switch - VM2.
This patch fixes two panic.
First panic:
root at current:~ # ifconfig vlan create vlan 1024 vlandev vtnet0 up
vlan0
root at current:~ # pkt-gen -i vlan0 -f tx
611.182750 main [2889] interface is vlan0
611.183004 main [3011] using default burst size: 512
611.183021 main [3019] running on 1 cpus (have 4)
611.185667 extract_ip_range [471] range is 10.0.611.186530 [1129] generic_netmap_attach Emulated adapter for vlan0 created (prev was 0)
0.1:1234 to 10.0.0.1:1234
611.185897 extract_ip_range [471] range is 10.1.0.1:1234 to 10.1.0.1:1234
611.185980 nm_open [858] overriding ARG1 0
611.186215 nm_open [862] overriding ARG2 0
611.186227 nm_open [866] overriding ARG3 0
611.186237 nm_open [870] overriding RING_CFG
611.186248 nm_open [879] overriding ifname vlan0 ringid 0x0 flags 0x8001
611.385901 [ 320] generic_netmap_register Emulated adapter for vlan0 activated
611.387781 main [3117] mapped 334980KB at 0x800e00000
Sending on netmap:vlan0: 1 queues, 1 threads and 1 cpus.
10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> ff:ff:ff:ff:ff:ff)
611.389515 main [3224] Sending 512 packets every 0.000000000 s
611.389636 start_threads [2549] Wait 2 secs for phy reset
613.435285 start_threads [2551] Ready...
613.435737 sender_body [1580] start, fd 3 main_fd 3
613.436545 sender_body [1638] frags 1 frag_
Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address = 0x28
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff80cd5f2e
stack pointer = 0x28:0xfffffe001971d520
frame pointer = 0x28:0xfffffe001971d550
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 732 (pkt-gen)
trap number = 12
panic: page fault
cpuid = 1
time = 1561989613
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe001971d1e0
vpanic() at vpanic+0x19d/frame 0xfffffe001971d230
panic() at panic+0x43/frame 0xfffffe001971d290
trap_fatal() at trap_fatal+0x39c/frame 0xfffffe001971d2f0
trap_pfault() at trap_pfault+0x62/frame 0xfffffe001971d340
trap() at trap+0x2b4/frame 0xfffffe001971d450
calltrap() at calltrap+0x8/frame 0xfffffe001971d450
--- trap 0xc, rip = 0xffffffff80cd5f2e, rsp = 0xfffffe001971d520, rbp = 0xfffffe001971d550 ---
ether_8021q_frame() at ether_8021q_frame+0x2e/frame 0xfffffe001971d550
vlan_transmit() at vlan_transmit+0xd4/frame 0xfffffe001971d5c0
nm_os_generic_xmit_frame() at nm_os_generic_xmit_frame+0x48/frame 0xfffffe001971d5d0
generic_netmap_txsync() at generic_netmap_txsync+0x25d/frame 0xfffffe001971d6a0
netmap_poll() at netmap_poll+0x324/frame 0xfffffe001971d770
freebsd_netmap_poll() at freebsd_netmap_poll+0x32/frame 0xfffffe001971d7a0
devfs_poll_f() at devfs_poll_f+0x71/frame 0xfffffe001971d7f0
kern_poll() at kern_poll+0x3d9/frame 0xfffffe001971d970
sys_poll() at sys_poll+0x50/frame 0xfffffe001971d990
amd64_syscall() at amd64_syscall+0x276/frame 0xfffffe001971dab0
fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe001971dab0
--- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8004943aa, rsp = 0x7fffdfffde48, rbp = 0x7fffdfffde80 ---
KDB: enter: panic
[ thread pid 732 tid 100137 ]
Stopped at kdb_enter+0x3b: movq $0,kdb_why
db>
Netmap generic directly calls the if_transmit method of the underlying driver (https://svnweb.freebsd.org/base/head/sys/dev/netmap/netmap_freebsd.c?revision=348022&view=markup#l455). Panic occurs at https://svnweb.freebsd.org/base/head/sys/net/if_ethersubr.c?revision=348254&view=markup#l1371 due to the fact that the current VNET is not set. The panic disappears if the VIMAGE option is disabled in the config.
To correct this panic, set the current VNET to the driver VNET.
The second panic:
root at current:~ # pkt-gen -i vlan0 -f tx
372.565904 main [2889] interface is vlan0
372.566203 main [3011] using default burst size: 512
372.566513 main [3019] running on 1 cpus (have 4)
372.568995 extract_ip_range [471] range is 10.0.0.1:1234 to 10.0.0.1:1234
372.569272 extract_ip_range [471] ran372.571191 [1129] generic_netmap_attach Emulated adapter for vlan0 created (prev was 0)
ge is 10.1.0.1:1234 to 10.1.0.1:1234
372.569632 nm_open [858] overriding ARG1 0
372.569930 nm_open [862] overriding ARG2 0
372.570231 nm_open [866] overriding ARG3 0
372.570534 nm_open [870] overriding RING_CFG
372.570863 nm_open [879] overriding ifname vlan0 ringid 0x0 flags 0x8001
372.785305 [ 320] generic_netmap_register Emulated adapter for vlan0 activated
372.787260 main [3117] mapped 334980KB at 0x800e00000
Sending on netmap:vlan0: 1 queues, 1 threads and 1 cpus.
10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> ff:ff:ff:ff:ff:ff)
372.788704 main [3224] Sending 512 packets every 0.000000000 s
372.789062 start_threads [2549] Wait 2 secs for phy reset
374.865353 start_threads [2551] Ready...
374.866365 sender_body [1580] start, fd 3 main_fd 3
374.866666 sender_body [1638] frags 1 frag_size 60
panic: vtnet_txq_encap: no mbuf packet header!
cpuid = 0
time = 1561994374
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0019732380
vpanic() at vpanic+0x19d/frame 0xfffffe00197323d0
panic() at panic+0x43/frame 0xfffffe0019732430
vtnet_txq_encap() at vtnet_txq_encap+0x3ba/frame 0xfffffe00197324a0
vtnet_txq_mq_start_locked() at vtnet_txq_mq_start_locked+0x126/frame 0xfffffe0019732500
vtnet_txq_mq_start() at vtnet_txq_mq_start+0x70/frame 0xfffffe0019732540
vlan_transmit() at vlan_transmit+0xe6/frame 0xfffffe00197325b0
nm_os_generic_xmit_frame() at nm_os_generic_xmit_frame+0x78/frame 0xfffffe00197325d0
generic_netmap_txsync() at generic_netmap_txsync+0x25d/frame 0xfffffe00197326a0
netmap_poll() at netmap_poll+0x324/frame 0xfffffe0019732770
freebsd_netmap_poll() at freebsd_netmap_poll+0x32/frame 0xfffffe00197327a0
devfs_poll_f() at devfs_poll_f+0x71/frame 0xfffffe00197327f0
kern_poll() at kern_poll+0x3d9/frame 0xfffffe0019732970
sys_poll() at sys_poll+0x50/frame 0xfffffe0019732990
amd64_syscall() at amd64_syscall+0x276/frame 0xfffffe0019732ab0
fast_syscall_common() at fast_syscall_common+0x101/frame 0xfffffe0019732ab0
--- syscall (209, FreeBSD ELF64, sys_poll), rip = 0x8004943aa, rsp = 0x7fffdfffde48, rbp = 0x7fffdfffde80 ---
KDB: enter: panic
[ thread pid 779 tid 100113 ]
Stopped at kdb_enter+0x3b: movq $0,kdb_why
db>
Netmap uses a preallocated array of mbuf's to send packets through the underlying driver. These mbuf's are allocated and initialized during the creation of the netmap adapter, in particular, the M_PKTHDR flag is set for them. After completing sending, netmap tries to reuse these mbuf's, but some network drivers (if_vlan, if_vxlan) may reset the M_PKTHDR flag. This leads to a panic on M_ASSERTPKTHDR (m) at line: https://svnweb.freebsd.org/base/head/sys/dev/virtio/network/if_vtnet.c?revision=348599&view=markup#l2201
To fix the second panic, restore M_PKTHDR flag before call if_transmit method.
TEST PLAN
pkt-gen test through vlan beetween real machines over physical network to check first panic:
1. Machine
# ifconfig vlan create vlan10 vlandev ix0 up
vlan0
# pkt-gen -i vlan0 -f tx
2. Machine
# ifconfig vlan create vlan10 vlandev ix0 up
vlan0
# pkt-gen -i vlan0 -f rx
pkt-gen test beetween VM's on same machine to check the second panic:
VM1:
# ifconfig vlan create vlan10 vlandev vtnet0 up
vlan0
# pkt-gen -i vlan0 -f tx
VM2:
# ifconfig vlan create vlan10 vlandev vtnet0 up
vlan0
# pkt-gen -i vlan0 -f rx
REVISION DETAIL
https://reviews.freebsd.org/D20824
AFFECTED FILES
sys/dev/netmap/netmap_freebsd.c
CHANGE DETAILS
diff --git a/sys/dev/netmap/netmap_freebsd.c b/sys/dev/netmap/netmap_freebsd.c
--- a/sys/dev/netmap/netmap_freebsd.c
+++ b/sys/dev/netmap/netmap_freebsd.c
@@ -444,6 +444,7 @@
m->m_ext.ext_size = len;
#endif /* __FreeBSD_version >= 1100000 */
+ m->m_flags |= M_PKTHDR;
m->m_len = m->m_pkthdr.len = len;
/* mbuf refcnt is not contended, no need to use atomic
@@ -452,7 +453,9 @@
M_HASHTYPE_SET(m, M_HASHTYPE_OPAQUE);
m->m_pkthdr.flowid = a->ring_nr;
m->m_pkthdr.rcvif = ifp; /* used for tx notification */
+ CURVNET_SET(ifp->if_vnet);
ret = NA(ifp)->if_transmit(ifp, m);
+ CURVNET_RESTORE();
return ret ? -1 : 0;
}
EMAIL PREFERENCES
https://reviews.freebsd.org/settings/panel/emailpreferences/
To: aleksandr.fedorov_itglobal.com, vmaffione, jhb, bz
Cc: krion, evgueni.gavrilov_itglobal.com, freebsd-net-list
-------------- next part --------------
A non-text attachment was scrubbed...
Name: D20824.59258.patch
Type: text/x-patch
Size: 638 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-net/attachments/20190701/40842437/attachment.bin>
More information about the freebsd-net
mailing list