bhyve terribly slow when using tagged vlan

Reply: jason_a_tubnor.net: "RE: bhyve terribly slow when using tagged vlan "
Go to: [ bottom of page ] [ top of archives ] [ this month ]
From: Daniel Braniss <danny_at_cs.huji.ac.il>
Date: Fri, 14 Jan 2022 10:50:17 UTC
hi,
the host is a very new Dell PowerEdge R6525 running FreeBSD 13.0,
I have connected ixl0 to an untagged link and ixl1 to a tagged link

I have created 2 bhyve clients, identical, except for the links, one is using ixl0 and the other ixl1.340

the untagged boots diskless in about 40 seconds, while the one using the tagged one takes 12 minutes!

I compared their dmesgs and they are identical (except for minor diffs like mac address)

any ideas?

danny

some more data:

sa# uname -a
FreeBSD sa 13.0-STABLE FreeBSD 13.0-STABLE #4 huji-13-736697896: Mon Dec 13 14:03:04 IST 2021     danny@hms-00:/home/obj/hms-00/h/rnd/git/stable/13/amd64.amd64/sys/HUJI  amd64
sa# ifconfig
ixl0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
       options=4a500b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,NOMAP>
       ether b4:96:91:97:08:14
       inet 132.65.116.32 netmask 0xfffff000 broadcast 132.65.127.255
       media: Ethernet autoselect (10Gbase-T <full-duplex>)
       status: active
       nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ixl1: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
       options=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
       ether b4:96:91:97:08:15
       media: Ethernet autoselect (10Gbase-T <full-duplex>)
       status: active
       nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bge0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
       options=c019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
       ether b0:7b:25:dd:ff:9c
       media: Ethernet autoselect
       nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bge1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
       options=c019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
       ether b0:7b:25:dd:ff:9d
       media: Ethernet autoselect
       nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
       options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
       inet6 ::1 prefixlen 128
       inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
       inet 127.0.0.1 netmask 0xff000000
       groups: lo
       nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
ixl1.340: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
       options=4200401<RXCSUM,LRO,RXCSUM_IPV6,NOMAP>
       ether b4:96:91:97:08:15
       inet 132.65.80.251 netmask 0xfffff000 broadcast 132.65.95.255
       groups: vlan
       vlan: 340 vlanproto: 802.1q vlanpcp: 0 parent interface: ixl1
       media: Ethernet autoselect (10Gbase-T <full-duplex>)
       status: active
       nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ue0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
       ether b0:7b:25:e5:dc:57
       nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vm-servers: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
       ether 0a:91:de:0e:f6:26
       id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
       maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
       root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
       member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
               ifmaxaddr 0 port 10 priority 128 path cost 2000000
       member: ixl0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
               ifmaxaddr 0 port 1 priority 128 path cost 2000
       groups: bridge vm-switch viid-d5539@
       nd6 options=9<PERFORMNUD,IFDISABLED>
vm-system: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
       ether 42:5b:f8:8f:72:00
       id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
       maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
       root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
       member: tap1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
		                ifmaxaddr 0 port 11 priority 128 path cost 2000000
       member: ixl1.340 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
               ifmaxaddr 0 port 6 priority 128 path cost 2000
       groups: bridge vm-switch viid-54b53@
       nd6 options=9<PERFORMNUD,IFDISABLED>
tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
       description: vmnet-bhv-08-0-servers
       options=80000<LINKSTATE>
       ether 58:9c:fc:10:82:13
       groups: tap vm-port
       media: Ethernet autoselect
       status: active
       nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
       Opened by PID 9357
tap1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
       description: vmnet-bhv-09-0-system
       options=80000<LINKSTATE>
       ether 58:9c:fc:10:ff:a2
       groups: tap vm-port
       media: Ethernet autoselect
       status: active
       nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>