From nobody Sun Jul 21 04:53:33 2024 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WRWJY69Drz5QwrS for ; Sun, 21 Jul 2024 04:53:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WRWJY3tyXz4RCC for ; Sun, 21 Jul 2024 04:53:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1721537613; a=rsa-sha256; cv=none; b=ljGTlQr5NHXlN47wp1Ho/zXoJYnfy2M0O+gKHS4xaPBGL0qch+L/+KHtNBv2p0iCwpuoiM ISkxS8qRjxvwEg2ZzDyBLIc37lxEXeuR8NaWnM8CAyg6rFDygKJVn4gIYguEYjzbkyYCnM brQpVmzKbofV+YziVw+/vx1WnoGsg4toC41HpKBW63NeIKHFWUaqEYO+DxAJlzW4QoeLdk VWXckwpWltkHIwq2oB3O6bNsdf/A6JxJKcaJS1fFUlF1LYo/B0rxJ8rfGexXrhWIiyt/xA hHJCVXp+E3x81CRbj2gHoRzuZ6SlSLirI9WiDoGixZz+Ltt1HwvQ0H9OZsFt0Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721537613; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=lt/4MTSYiZchBhcZK6mEqEpv/cVMmeEq+gsB21WqHhw=; b=EFl56qpZlZxrl8IMf8k4o9nct3p5DRPY6SVNqJmfYjrrQPozoF8H04M/8cswM+Gc7Gk1vq ptlBonCe76N6FeO/UGSTmpsW1jK7vgQsNQN1P6JKgUA8iB8fGNP/sK8C9pENo9DCTgokBl puXJYwkgSBbOBIWUCE1ZWsi6AP8zm60q0//YyuduHxRtLLHl/H7ELg9Yvcsr28INyOzgAZ /gMzQ+tj/9X8SRtnXQxQiNORjoaxlCq7ggypip1sgxVjj60v8jfUzCv58DVbeAT8zbcmQS 9rUdHsxbQ6ynxJCTnLg5EawFZiKfkh8FXd0Lv58GCV7X2yNKD+77Hmu04VtfMA== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WRWJY3BrPzmMr for ; Sun, 21 Jul 2024 04:53:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 46L4rXFw026075 for ; Sun, 21 Jul 2024 04:53:33 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 46L4rXw5026074 for bugs@FreeBSD.org; Sun, 21 Jul 2024 04:53:33 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 280386] if_bridge throws output errors under load Date: Sun, 21 Jul 2024 04:53:33 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 14.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: pascal.guitierrez@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@FreeBSD.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D280386 Bug ID: 280386 Summary: if_bridge throws output errors under load Product: Base System Version: 14.1-RELEASE Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: pascal.guitierrez@gmail.com I'm able to easily make if_bridge spew thousands of output errors with a re= ally simple config and using iperf3. this results into random "no buffer space" errors on the host and disruptio= ns to services. only reproduceable when using a bridge - if no bridge involved there are no errors or disruptions. OS is 14.1-RELEASE-p2 on amd64. /etc/rc.conf: cloned_interfaces=3D"bridge0" ifconfig_bridge0=3D"addm igb0 192.168.0.150/24 up" ifconfig_igb0=3D"-txcsum -txcsum6 -tso up" (the reason for turning off certain flags is to prevent link-flap when the = VM is disconnected from the bridge) netstat -i: (note the Oerrs field below) Name Mtu Network Address Ipkts Ierrs Idrop O= pkts Oerrs Coll igb0 1500 5c:ed:8c:e9:c2:48 50259033 0 0 5316= 6489 0 0 igb1* 1500 5c:ed:8c:e9:c2:49 0 0 0 = 0 0 0 igb2* 1500 5c:ed:8c:e9:c2:4a 0 0 0 = 0 0 0 igb3* 1500 5c:ed:8c:e9:c2:4b 0 0 0 = 0 0 0 lo0 16384 lo0 24959 0 0 2= 4959 0 0 lo0 - localhost localhost 0 - - = 0 - - lo0 - fe80::%lo0/64 fe80::1%lo0 0 - - = 0 - - lo0 - your-net localhost 24959 - - 2= 4959 - - bridge0 1500 58:9c:fc:00:07:00 103262907 0 0 10314= 1654 130543 0 bridge0 - 192.168.0.0/24 192.168.0.150 235137 - - 22= 5757 - - ue0 1500 72:84:d1:bf:ad:2f 3587 0 0 = 3588 0 0 ue0 - 16.1.15.0/30 16.1.15.2 3308 - - = 3308 - - tap0 1500 58:9c:fc:10:f3:0f 53003359 0 0 4824= 9364 0 0 can reproduce across multiple systems of varying compute and network power (have tested on ix* NICs with similar results) iperf3 cmd line used: iperf3 -P4 -c 192.168.0.5 -t 60 remote iperf3 system is a windows server, network is gigabit. errors occur only when testing in one direction (FreeBSD bridge client -> remote system server). if i don't use if_bridge, ie: ifconfig_igb0=3D"-txcsum -txcsum6 -tso 192.168.0.150/24 up" then no errors or dropped packets are seen at all. is my config correct? are there known issues with if_bridge in freebsd? dmesg below: ---<>--- Copyright (c) 1992-2023 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 14.1-RELEASE releng/14.1-n267679-10e31f0946d8 GENERIC amd64 FreeBSD clang version 18.1.5 (https://github.com/llvm/llvm-project.git llvmorg-18.1.5-0-g617a15a9eac9) VT(efifb): resolution 1024x768 CPU: Intel(R) Xeon(R) E-2314 CPU @ 2.80GHz (2808.00-MHz K8-class CPU) Origin=3D"GenuineIntel" Id=3D0xa0671 Family=3D0x6 Model=3D0xa7 Steppi= ng=3D1 =20 Features=3D0xbfebfbff =20 Features2=3D0x7ffafbff AMD Features=3D0x2c100800 AMD Features2=3D0x121 Structured Extended Features=3D0xf2bf67ef Structured Extended Features2=3D0x40405f5e Structured Extended Features3=3D0xbc000410 XSAVE Features=3D0xf IA32_ARCH_CAPS=3D0x2023c6b VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory =3D 17179869184 (16384 MB) avail memory =3D 16503263232 (15738 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: unblocking device. ioapic0 irqs 0-119 Launching APs: 2 3 1 random: entropy device external interface kbd1 at kbdmux0 efirtc0: efirtc0: registered as a time-of-day clock, resolution 1.000000s smbios0: at iomem 0x72fc8000-0x72fc801e smbios0: Version: 3.3, BCD Revision: 3.3 aesni0: acpi0: acpi0: Power Button (fixed) attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 24000000 Hz quality 950 Event timer "HPET" frequency 24000000 Hz quality 550 Event timer "HPET1" frequency 24000000 Hz quality 440 Event timer "HPET2" frequency 24000000 Hz quality 440 Event timer "HPET3" frequency 24000000 Hz quality 440 Event timer "HPET4" frequency 24000000 Hz quality 440 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x508-0x50b on acpi0 apei0: on acpi0 acpi_syscontainer0: on acpi0 pcib0: port 0xcf8-0xcff numa-domain 0 on acpi0 pci0: numa-domain 0 on pcib0 xhci0: mem 0x4000000000-0x400000fff= f at device 20.0 numa-domain 0 on pci0 xhci0: 32 bytes context size, 64-bit DMA usbus0 numa-domain 0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci0: at device 20.2 (no driver attached) pci0: at device 22.0 (no driver attached) pci0: at device 22.4 (no driver attached) ahci0: port 0x3040-0x3047,0x3048-0x304b,0x3020-0x303f mem 0x80580000-0x80581fff,0x80584000-0x805840ff,0x80583000-0x805837ff at de= vice 23.0 numa-domain 0 on pci0 ahci0: AHCI v1.31 with 4 6Gbps ports, Port Multiplier not supported ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahcich4: at channel 4 on ahci0 ahcich5: at channel 5 on ahci0 ahciem0: on ahci0 pcib1: at device 27.0 numa-domain 0 on pci0 pci1: numa-domain 0 on pcib1 igb0: port 0x2060-0x207f mem 0x80300000-0x803fffff,0x8040c000-0x8040ffff at device 0.0 numa-domain 0 on = pci1 igb0: EEPROM V1.63-0 Option ROM V1-b3310-p0 eTrack 0x8000119d igb0: Using 1024 TX descriptors and 1024 RX descriptors igb0: Using 4 RX queues 4 TX queues igb0: Using MSI-X interrupts with 5 vectors igb0: Ethernet address: 5c:ed:8c:e9:c2:48 igb0: netmap queues/slots: TX 4/1024, RX 4/1024 igb1: port 0x2040-0x205f mem 0x80200000-0x802fffff,0x80408000-0x8040bfff at device 0.1 numa-domain 0 on = pci1 igb1: EEPROM V1.63-0 Option ROM V1-b3310-p0 eTrack 0x8000119d igb1: Using 1024 TX descriptors and 1024 RX descriptors igb1: Using 4 RX queues 4 TX queues igb1: Using MSI-X interrupts with 5 vectors igb1: Ethernet address: 5c:ed:8c:e9:c2:49 igb1: netmap queues/slots: TX 4/1024, RX 4/1024 igb2: port 0x2020-0x203f mem 0x80100000-0x801fffff,0x80404000-0x80407fff at device 0.2 numa-domain 0 on = pci1 igb2: EEPROM V1.63-0 Option ROM V1-b3310-p0 eTrack 0x8000119d igb2: Using 1024 TX descriptors and 1024 RX descriptors igb2: Using 4 RX queues 4 TX queues igb2: Using MSI-X interrupts with 5 vectors igb2: Ethernet address: 5c:ed:8c:e9:c2:4a igb2: netmap queues/slots: TX 4/1024, RX 4/1024 igb3: port 0x2000-0x201f mem 0x80000000-0x800fffff,0x80400000-0x80403fff at device 0.3 numa-domain 0 on = pci1 igb3: EEPROM V1.63-0 Option ROM V1-b3310-p0 eTrack 0x8000119d igb3: Using 1024 TX descriptors and 1024 RX descriptors igb3: Using 4 RX queues 4 TX queues igb3: Using MSI-X interrupts with 5 vectors igb3: Ethernet address: 5c:ed:8c:e9:c2:4b igb3: netmap queues/slots: TX 4/1024, RX 4/1024 pcib2: at device 27.6 numa-domain 0 on pci0 pci2: numa-domain 0 on pcib2 pcib3: at device 29.0 numa-domain 0 on pci0 pci3: numa-domain 0 on pcib3 vgapci0: mem 0xa4000000-0xa4ffffff,0xa5b98000-0xa5b9bfff,0xa5000000-0xa57fffff at device= 0.1 numa-domain 0 on pci3 vgapci0: Boot video device ehci0: mem 0xa5b9c000-0xa5b9c0ff at dev= ice 0.4 numa-domain 0 on pci3 usbus1: EHCI version 1.0 usbus1 numa-domain 0 on ehci0 usbus1: 480Mbps High Speed USB v2.0 pcib4: at device 29.1 numa-domain 0 on pci0 pci4: numa-domain 0 on pcib4 pcib5: at device 29.2 numa-domain 0 on pci0 pci5: numa-domain 0 on pcib5 pcib6: at device 29.3 numa-domain 0 on pci0 pci6: numa-domain 0 on pcib6 pcib7: at device 29.4 numa-domain 0 on pci0 pci7: numa-domain 0 on pcib7 isab0: at device 31.0 numa-domain 0 on pci0 isa0: numa-domain 0 on isab0 pci0: at device 31.5 (no driver attached) cpu0: numa-domain 0 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 uart0: port 0x3f8-0x3ff irq 4 f= lags 0x10 on acpi0 vga0: at port 0x3b0-0x3bb iomem 0xb0000-0xb7fff pnpid PNP= 0900 on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbdc0: non-PNP ISA device will be removed from GENERIC in FreeBSD 15. hwpstate_intel0: numa-domain 0 on cpu0 hwpstate_intel1: numa-domain 0 on cpu1 hwpstate_intel2: numa-domain 0 on cpu2 hwpstate_intel3: numa-domain 0 on cpu3 Timecounter "TSC-low" frequency 1404004245 Hz quality 1000 Timecounters tick every 1.000 msec ZFS filesystem version: 5 ZFS storage pool version: features support (5000) ugen1.1: <(0x103c) EHCI root HUB> at usbus1 Trying to mount root from zfs:zroot/ROOT/default []... ugen0.1: at usbus0 uhub0 numa-domain 0 on usbus0 uhub0: on usbus0 uhub1 numa-domain 0 on usbus1 uhub1: <(0x103c) EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1 ada0 at ahcich2 bus 0 scbus0 target 0 lun 0 ada0: ACS-4 ATA SATA 3.x device ada0: Serial Number WS255C4V ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 3815447MB (7814037168 512 byte sectors) ada1 at ahcich3 bus 0 scbus1 target 0 lun 0 ada1: ACS-4 ATA SATA 3.x device ada1: Serial Number WS255BGF ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 3815447MB (7814037168 512 byte sectors) ada2 at ahcich4 bus 0 scbus2 target 0 lun 0 ada2: ACS-2 ATA SATA 3.x device ada2: Serial Number BTWA518300MH240AGN ada2: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes) ada2: Command Queueing enabled ada2: 228936MB (468862128 512 byte sectors) ada3 at ahcich5 bus 0 scbus3 target 0 lun 0 ada3: ACS-2 ATA SATA 3.x device ada3: Serial Number BTWA51740BM1240AGN ada3: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes) ada3: Command Queueing enabled ada3: 228936MB (468862128 512 byte sectors) ses0 at ahciem0 bus 0 scbus4 target 0 lun 0 ses0: SEMB S-E-S 2.00 device ses0: SEMB SES Device ses0: ada0,pass0 in 'Slot 02', SATA Slot: scbus0 target 0 ses0: ada1,pass1 in 'Slot 03', SATA Slot: scbus1 target 0 ses0: ada2,pass2 in 'Slot 04', SATA Slot: scbus2 target 0 ses0: ada3,pass3 in 'Slot 05', SATA Slot: scbus3 target 0 uhub0: 22 ports with 22 removable, self powered Root mount waiting for: usbus0 usbus1 Root mount waiting for: usbus0 usbus1 usb_msc_auto_quirk: UQ_MSC_NO_GETMAXLUN set for USB mass storage device Gen= eric Mass Storage Device (0x14cd:0x1212) usb_msc_auto_quirk: UQ_MSC_NO_PREVENT_ALLOW set for USB mass storage device Generic Mass Storage Device (0x14cd:0x1212) usb_msc_auto_quirk: UQ_MSC_NO_SYNC_CACHE set for USB mass storage device Generic Mass Storage Device (0x14cd:0x1212) ugen0.2: at usbus0 umass0 numa-domain 0 on uhub0 umass0: on usbus0 umass0: SCSI over Bulk-Only; quirks =3D 0xc100 umass0:5:0: Attached to scbus5 da0 at umass-sim0 bus 0 scbus5 target 0 lun 0 da0: Removable Direct Access SCSI device da0: Serial Number 121220160204 da0: 40.000MB/s transfers da0: 3968MB (8126464 512 byte sectors) da0: quirks=3D0x2 uhub1: 8 ports with 8 removable, self powered Root mount waiting for: usbus0 usbus1 ugen1.2: at usbus1 uhub_explore: illegal enable change, port 4 usb_msc_auto_quirk: UQ_MSC_NO_GETMAXLUN set for USB mass storage device Gen= eric Mass Storage Device (0x14cd:0x1212) usb_msc_auto_quirk: UQ_MSC_NO_PREVENT_ALLOW set for USB mass storage device Generic Mass Storage Device (0x14cd:0x1212) usb_msc_auto_quirk: UQ_MSC_NO_SYNC_CACHE set for USB mass storage device Generic Mass Storage Device (0x14cd:0x1212) ugen0.3: at usbus0 umass1 numa-domain 0 on uhub0 umass1: on usbus0 umass1: SCSI over Bulk-Only; quirks =3D 0xc100 umass1:6:1: Attached to scbus6 da1 at umass-sim1 bus 1 scbus6 target 0 lun 0 da1: Removable Direct Access SCSI device da1: Serial Number 121220160204 da1: 40.000MB/s transfers da1: 3968MB (8126464 512 byte sectors) da1: quirks=3D0x2 ugen0.4: at usbus0 uhub2 numa-domain 0 on uhub0 uhub2: on usbus0 uhub2: 2 ports with 1 removable, self powered ichsmb0: port 0xefa0-0xefbf mem 0x4000014000-0x40000140ff at device 31.4 numa-domain 0 on pci0 smbus0: numa-domain 0 on ichsmb0 driver bug: Unable to set devclass (class: atrtc devname: (unknown)) acpi_wmi0: on acpi0 acpi_wmi0: cannot find EC device bridge0: Ethernet address: 58:9c:fc:00:07:00 igb0: link state changed to UP lo0: link state changed to UP igb0: link state changed to DOWN bridge0: link state changed to UP igb0: promiscuous mode enabled cdceem0 numa-domain 0 on uhub1 cdceem0: on usbus1 ue0: on cdceem0 ue0: Ethernet address: 72:84:d1:bf:ad:2f igb0: link state changed to UP driver bug: Unable to set devclass (class: atrtc devname: (unknown)) tap0: Ethernet address: 58:9c:fc:10:f3:0f tap0: promiscuous mode enabled tap0: link state changed to UP driver bug: Unable to set devclass (class: atrtc devname: (unknown)) ipmi0: port 0xca2-0xca3 on acpi0 ipmi0: KCS mode found at io 0xca2 on acpi ipmi0: IPMI device rev. 3, firmware rev. 3.01, version 2.0, device support = mask 0x9f ipmi0: Number of channels 2 ipmi0: Attached watchdog ipmi0: Establishing power cycle handler --=20 You are receiving this mail because: You are the assignee for the bug.=