[Bug 278245] sendfile to bhyve VM on same host fails with MEXTPG

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=278245

            Bug ID: 278245
           Summary: sendfile to bhyve VM on same host fails with MEXTPG
           Product: Base System
           Version: 14.0-RELEASE
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: djk-freebsdbugs@needful.ca

In debugging an issue with an nginx reverse proxy, I found an issue with
sendfile, MEXTPG, and one of more of bridge, tap, and bhyve.  I am on FreeBSD
14-CURRENT-p6.

To replicate this issue, I used the following Python test harness:

import socket    
HOST = 'target.host'
PORT = 50007    
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s, open("blah.txt",
"rb") as f:
    s.connect((HOST, PORT))
    s.sendfile(f)

In this case, target.host is a Debian bhyve VM running on the same host.  On
the target, something like 'nc -l -p 50007' can be run to receive the data.

On the networking side, I've got:

# ifconfig em0
em0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP>
metric 0 mtu 1500
       
options=a520b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,HWSTATS>
        ether 6c:4b:90:1f:e9:a8
        inet 192.168.11.15 netmask 0xffffff00 broadcast 192.168.11.255
        inet6 fe80::6e4b:90ff:fe1f:e9a8%em0 prefixlen 64 scopeid 0x1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
# ifconfig vm-public
vm-public: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP>
metric 0 mtu 1500
        options=0
        ether 3a:47:10:77:5b:4d
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: tap3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 7 priority 128 path cost 2000000
        member: tap2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 6 priority 128 path cost 2000000
        member: tap1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 5 priority 128 path cost 2000000
        member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 2000000
        member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 20000
        groups: bridge vm-switch viid-4c918@
        nd6 options=9<PERFORMNUD,IFDISABLED>


When MEXTPG is enabled on the em0 interface, the test fails when sending a file
larger than 224 bytes.  When MEXTPG is enabled the test is successful.  The
test is successful when sending to a VM on a different host, so the bhyve VM is
a requirement to replicate the issue.

https://www.reddit.com/r/freebsd/comments/1bxebaz/more_vm_networking_weirdness/
has a lot more detail and some narrowing down of the problem.

-- 
You are receiving this mail because:
You are the assignee for the bug.