From nobody Wed Mar 29 05:05:08 2023 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PmZHc03Bxz41ynd for ; Wed, 29 Mar 2023 05:05:16 +0000 (UTC) (envelope-from zlei@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PmZHb5sHWz4JH3; Wed, 29 Mar 2023 05:05:15 +0000 (UTC) (envelope-from zlei@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1680066315; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=opwVvHZZqVYFltrFswPlfwyvkONUuy1VULwY9xQ6Mvw=; b=sFKVwxeRR1gAn7+DwaKtHyOp8c0Hoe2QMH1zPgRdAi4fUFUFpHhDIYXWVg5mJQhDWYDQ6C 8nfNzxNzdwofVqRcjtMUZAPa4oRoMlGLWAaWgfjvnW+aYM1Dvd0DFfYF1PNiuUf+62H8M+ a93TYSvpvaJ8BTm8n1VIG65/oqVSINHO5uQSusf+Y83wus8LD3Ni4B4JyOt3jrbkGZCS7k /cbY5ZvbfC0k1P0XrtfGsY/VSPJeNEhn40OEVPggLlIvK6okYSthylSLvYgoMluIMjH8SS smkDLa8dM8vhLKbJNN6JIGRO/r6+kn4A+LRF2OxZVKUnMEWd212UZqtzwYck/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1680066315; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=opwVvHZZqVYFltrFswPlfwyvkONUuy1VULwY9xQ6Mvw=; b=R9vldK42/TIoQLfm92S04V7/WY15kCg+AwUDDXO4gVR/x99x2hdYfyzVH+mYyv+Fh0F5XS tfyXTJFGOcE/b3A4ku3Z+zABFCO7vtyJPSxCBY5QW0RyEfZDoCXxfPXkUnKYjvukdeQTrn Us3ls4jhBEvmaFhevrWPGMv1vWos3S8JauFirDsOeYkBYCnf95ciV8kGDNdram3OMvKsh2 awwDa1rxS8vXBiPAvuPr34cXowT61jyXZxhRSe6ycOoLOf5UXMDdcM0JxuSRXR9Ei73bJx sNmfwl0G1SbK8SpwiNI2gGXncgBOw2CdMIJZ1zpoRpEH4RrQvSb5dGBgEFYWsA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1680066315; a=rsa-sha256; cv=none; b=wbN6K4Amt9i0XULu3pCxbGBuT3z8QQzdRLzdsLD+jM2HUkV5BetW6XEnrfIFiapmc38fJU aFjeoejPuiW+bkxf+DINThaOURkj5Dt42gFwsjV/KTzVdpo4llgQj6wvYxFT8S2AKY/8Sl SCKsBudKUSh5nOdP+hcAuoUxGoOpdhlguFAPjv228vruJZdf/OuzHvZb4wwaVrNtUQHjs+ YbnkhUhLMz1jRpzMssDflFnnaD8D4V1GOTOnFtEIPMzaidHguuP5yObtMewKZ4NdwAQ+aq bpI901Y6QVl3zrgecwV9a3eRuwTgkyoi3x+CIkPgNOGwi3q5sgX2i2+G9vNI2w== Received: from smtpclient.apple (ns1.oxydns.net [45.32.91.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: zlei/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4PmZHZ3Gfvzym8; Wed, 29 Mar 2023 05:05:14 +0000 (UTC) (envelope-from zlei@FreeBSD.org) From: Zhenlei Huang Message-Id: <4416034C-237B-4D8F-B04E-A22F2D56BF9C@FreeBSD.org> Content-Type: multipart/mixed; boundary="Apple-Mail=_688076BA-DFE4-44BD-8DCD-6F98600A8B12" List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Subject: Re: Help testing patch that may help diagnosing the PR 240106 Date: Wed, 29 Mar 2023 13:05:08 +0800 In-Reply-To: <4431A103-64AA-4EDF-9830-ED320161B75C@FreeBSD.org> Cc: freebsd-net@freebsd.org To: overwatch@lab.kyngin.net References: <4431A103-64AA-4EDF-9830-ED320161B75C@FreeBSD.org> X-Mailer: Apple Mail (2.3696.120.41.1.2) X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_688076BA-DFE4-44BD-8DCD-6F98600A8B12 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Mar 29, 2023, at 1:03 PM, Zhenlei Huang wrote: >=20 > Hi, >=20 > I write here so that the original PR 240106 is not polluted. >=20 > Can you please test the attached patch with bridge / lagg setup? >=20 > For long: >=20 > In https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D240106#c28 you = encountered > problem and I said: >=20 >> The IF_BRIDGE(4) seems to hide some thing to protect itself get = confused. > Actually IF_BRIDGE(4) has a learning mode. You can `man ifconfig` and = refer the > `Bridge Interface Parameters` section. >=20 > By default the learning mode of all bridge members is on, and the = bridge will > insert or update an entry to its (internal) forwarding table. When = unicast packets > come to the bridge member, the bridge will check if it is for itself, = if not then > the packets will be forwarded to one bridge member if a forwarding = entry is found. > While the magic is, if the bridge member to be forwarded is the = receiving one, then > the packets are silently discarded. >=20 > That's perfect fine, but will be hard to diagnose if user has wrong = network setup, > bridge loops e.g., or some other ones set duplicated ether address for = their nic, > or some bad guys / virus / trojans send spoofed packets on the wire. = Those are common > and I think it will be good if IF_BRIDGE(4) can emit logs so that the = symptoms will > be obvious and it will be easy to diagnose. >=20 >> If you can confirm, then please config you switch properly. The two = ports cc0 and cc1 connected should be in same link aggregation group. >=20 > If two ports (on physical switch), say 1 and 2, are not in same link = aggregation group, > then packets (typically broadcast ones) received on 1 will be = forwarded to 2, and > the lagg interface will be bounce-backed (from port 2) the packets it = send (to port 1). > If the lagg happenly be the member of IF_BRIDGE(4), then the bridge = will update > its forwarding entry as it learn mac address from lagg interface. >=20 > Here is a simple diagram, the arrow shows the flow of ARP request from = epair0a. >=20 > 11:22:33:44:55:66 [1] -> cc0 -> port 1 ->=20 > epair0a -> epair0b -> bridge0 -> lagg0 = physical-switch <-> host0 > <- <- cc1 <- port 2 <- =20 > [2] =20 >=20 > On [1] bridge0 will learn MAC 11:22:33:44:55:66 on port member epair0b = and add entry, > after [2] it will learn same MAC on port member lagg0 and update the = entry. Then > subsequent ARP reply (to 11:22:33:44:55:66, epair0a i.e.) sent from = host0 reach bridge0 > via lagg0. >=20 > Apparently bridge0 will dropped the ARP reply as it believes = 11:22:33:44:55:66 (epair0a) is > within segment of lagg0. >=20 >> I'll see if I can teach IF_BRIDGE(4) to emit warnings in case it get = ARP request packet sent from it self. >=20 > Attached patch will enable IF_BRIDGE(4) to emit logs about MAC address = port flapping. > Various hardware vendors have similar facilities. >=20 >=20 > Best regards, > Zhenlei >=20 Sorry forgot the patch. --Apple-Mail=_688076BA-DFE4-44BD-8DCD-6F98600A8B12 Content-Disposition: attachment; filename=0001-bridge-Log-MAC-address-port-flapping.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="0001-bridge-Log-MAC-address-port-flapping.patch" Content-Transfer-Encoding: quoted-printable =46rom=20787c9eb9d8065f85643ad32f2322e660ca897dfa=20Mon=20Sep=2017=20= 00:00:00=202001=0AFrom:=20Zhenlei=20Huang=20=0ADate:=20= Mon,=2027=20Mar=202023=2023:59:20=20+0800=0ASubject:=20[PATCH]=20bridge:=20= Log=20MAC=20address=20port=20flapping=0A=0A---=0A=20sys/net/if_bridge.c=20= |=2023=20+++++++++++++++++++++--=0A=201=20file=20changed,=2021=20= insertions(+),=202=20deletions(-)=0A=0Adiff=20--git=20= a/sys/net/if_bridge.c=20b/sys/net/if_bridge.c=0Aindex=20= 60a1683c74ae..41e009f19e87=20100644=0A---=20a/sys/net/if_bridge.c=0A+++=20= b/sys/net/if_bridge.c=0A@@=20-460,6=20+460,13=20@@=20= SYSCTL_INT(_net_link_bridge,=20OID_AUTO,=20allow_llz_overlap,=0A=20=20=20= =20=20"Allow=20overlap=20of=20link-local=20scope=20"=0A=20=20=20=20=20= "zones=20of=20a=20bridge=20interface=20and=20the=20member=20= interfaces");=0A=20=0A+/*=20log=20MAC=20port=20flapping=20*/=0A= +VNET_DEFINE_STATIC(int,=20log_mac_flap)=20=3D=201;=0A+#define=09= V_log_mac_flap=09VNET(log_mac_flap)=0A+SYSCTL_INT(_net_link_bridge,=20= OID_AUTO,=20log_mac_flap,=0A+=20=20=20=20CTLFLAG_RWTUN=20|=20= CTLFLAG_VNET,=20&VNET_NAME(log_mac_flap),=200,=0A+=20=20=20=20"Log=20MAC=20= port=20flapping");=0A+=0A=20struct=20bridge_control=20{=0A=20=09int=09= (*bc_func)(struct=20bridge_softc=20*,=20void=20*);=0A=20=09int=09= bc_argsize;=0A@@=20-2773,6=20+2780,7=20@@=20bridge_rtupdate(struct=20= bridge_softc=20*sc,=20const=20uint8_t=20*dst,=20uint16_t=20vlan,=0A=20=20= =20=20=20struct=20bridge_iflist=20*bif,=20int=20setflags,=20uint8_t=20= flags)=0A=20{=0A=20=09struct=20bridge_rtnode=20*brt;=0A+=09struct=20= bridge_iflist=20*obif;=0A=20=09int=20error;=0A=20=0A=20=09= BRIDGE_LOCK_OR_NET_EPOCH_ASSERT(sc);=0A@@=20-2796,7=20+2804,7=20@@=20= bridge_rtupdate(struct=20bridge_softc=20*sc,=20const=20uint8_t=20*dst,=20= uint16_t=20vlan,=0A=20=0A=20=09=09/*=20Check=20again,=20now=20that=20we=20= have=20the=20lock.=20There=20could=20have=0A=20=09=09=20*=20been=20a=20= race=20and=20we=20only=20want=20to=20insert=20this=20once.=20*/=0A-=09=09= if=20((brt=20=3D=20bridge_rtnode_lookup(sc,=20dst,=20vlan))=20!=3D=20= NULL)=20{=0A+=09=09if=20(bridge_rtnode_lookup(sc,=20dst,=20vlan)=20!=3D=20= NULL)=20{=0A=20=09=09=09BRIDGE_RT_UNLOCK(sc);=0A=20=09=09=09return=20= (0);=0A=20=09=09}=0A@@=20-2844,13=20+2852,24=20@@=20= bridge_rtupdate(struct=20bridge_softc=20*sc,=20const=20uint8_t=20*dst,=20= uint16_t=20vlan,=0A=20=09=09BRIDGE_RT_UNLOCK(sc);=0A=20=09}=0A=20=0A+=0A=20= =09if=20((brt->brt_flags=20&=20IFBAF_TYPEMASK)=20=3D=3D=20IFBAF_DYNAMIC=20= &&=0A-=09=20=20=20=20brt->brt_dst=20!=3D=20bif)=20{=0A+=09=20=20=20=20= (obif=20=3D=20brt->brt_dst)=20!=3D=20bif)=20{=0A=20=09=09= BRIDGE_RT_LOCK(sc);=0A=20=09=09brt->brt_dst->bif_addrcnt--;=0A=20=09=09= brt->brt_dst=20=3D=20bif;=0A=20=09=09brt->brt_dst->bif_addrcnt++;=0A=20=09= =09BRIDGE_RT_UNLOCK(sc);=0A+=0A+=09=09/*=20TODO=20also=20log=20aging=20= time=20?=20=20need=20(time_uptime=20-=20brt_create_time)=20*/=0A+=09=09= if=20(V_log_mac_flap)=0A+=09=09=09log(LOG_NOTICE,=20"%s:=20mac=20address=20= %02x:%02x:%02x:%02x:%02x:%02x=20vlan=20%d=20flaps=20from=20%s=20to=20= %s\n",=0A+=09=09=09=20=20=20=20sc->sc_ifp->if_xname,=0A+=09=09=09=20=20=20= =20brt->brt_addr[0],=20brt->brt_addr[1],=20brt->brt_addr[2],=0A+=09=09=09= =20=20=20=20brt->brt_addr[3],=20brt->brt_addr[4],=20brt->brt_addr[5],=0A= +=09=09=09=20=20=20=20brt->brt_vlan,=0A+=09=09=09=20=20=20=20= obif->bif_ifp->if_xname,=0A+=09=09=09=20=20=20=20= bif->bif_ifp->if_xname);=0A=20=09}=0A=20=0A=20=09if=20((flags=20&=20= IFBAF_TYPEMASK)=20=3D=3D=20IFBAF_DYNAMIC)=0A--=20=0A2.24.3=20(Apple=20= Git-128)=0A=0A= --Apple-Mail=_688076BA-DFE4-44BD-8DCD-6F98600A8B12 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii --Apple-Mail=_688076BA-DFE4-44BD-8DCD-6F98600A8B12--