From nobody Tue Jul 05 09:55:56 2022 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id DEA521D16445 for ; Tue, 5 Jul 2022 09:56:05 +0000 (UTC) (envelope-from SRS0=Ua7A=XK=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4LcdNN5rCpz4gVs for ; Tue, 5 Jul 2022 09:56:04 +0000 (UTC) (envelope-from SRS0=Ua7A=XK=klop.ws=ronald-lists@realworks.nl) Date: Tue, 5 Jul 2022 11:55:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1657014957; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=aMsUCuOJpu46AlObVCcc+TbB5BbDITk1+jZnDnjjlmQ=; b=nqG11UQbJ3Okm5D7Q2Cl4afeKnztmS74joGjTtb3yYrWKRwsaIUopnzl53BSB6iFUBITSG 0HYwyrR879xQcLEQb/g2v860ZVOG238AuQKlSko5KBz1fedIYbhHOcpqHo+NNZYEY/4BLV 54vsAykIuyvS4E6FmRy/YLRAHphlk9ri6wMsrJNqCmNXvOTpzuU+/t2ano/3XzH893p8cr eRDF787VLUojc9oMYF9ZA4TE7IUjYpSVbYd9D9oDgjDlxfK2R8FeUf6M44z4NKjMiZRSjJ 0DQvHsQiHx5mqPfF3cSwOQuRfe2SAj4ER3C4eR/ISc/5tadl7JEWzE9bjbSVRQ== From: Ronald Klop To: Mark Millard Cc: Karl Denninger , bob prohaska , freebsd-arm@freebsd.org Message-ID: <1645012198.135.1657014956867@localhost> In-Reply-To: <212C86C0-17DB-45F5-A59D-8BDC1932378E@yahoo.com> References: <20220704003639.GA1165@www.zefox.net> <8820A9EC-A25E-4D0A-9F8F-52114E58B66F@yahoo.com> <6c377413-9430-54d2-3f92-1215055ca30a@denninger.net> <20220704152834.GA1771@www.zefox.net> <7ce87eef-ded5-8b00-3f11-14407b8af78d@denninger.net> <20220704182526.GB1771@www.zefox.net> <212C86C0-17DB-45F5-A59D-8BDC1932378E@yahoo.com> Subject: duplicate MAC - Re: 13.1R problems on Pi3 List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_134_2008716236.1657014956808" X-Mailer: Realworks (613.97.a0c3c70) Importance: Normal X-Priority: 3 (Normal) X-Rspamd-Queue-Id: 4LcdNN5rCpz4gVs X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=klop.ws header.s=rw2 header.b=nqG11UQb; dmarc=pass (policy=quarantine) header.from=klop.ws; spf=pass (mx1.freebsd.org: domain of "SRS0=Ua7A=XK=klop.ws=ronald-lists@realworks.nl" designates 194.109.157.24 as permitted sender) smtp.mailfrom="SRS0=Ua7A=XK=klop.ws=ronald-lists@realworks.nl" X-Spamd-Result: default: False [-3.20 / 15.00]; ARC_NA(0.00)[]; FORGED_SENDER(0.30)[ronald-lists@klop.ws,SRS0=Ua7A=XK=klop.ws=ronald-lists@realworks.nl]; R_DKIM_ALLOW(-0.20)[klop.ws:s=rw2]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:194.109.157.0/24]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[klop.ws:+]; DMARC_POLICY_ALLOW(-0.50)[klop.ws,quarantine]; HAS_X_PRIO_THREE(0.00)[3]; NEURAL_HAM_SHORT(-1.00)[-0.999]; MLMMJ_DEST(0.00)[freebsd-arm]; FREEMAIL_TO(0.00)[yahoo.com]; RCVD_COUNT_ZERO(0.00)[0]; MIME_TRACE(0.00)[0:+,1:+,2:~]; MID_RHS_NOT_FQDN(0.50)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL]; FROM_NEQ_ENVFROM(0.00)[ronald-lists@klop.ws,SRS0=Ua7A=XK=klop.ws=ronald-lists@realworks.nl] X-ThisMailContainsUnwantedMimeParts: N ------=_Part_134_2008716236.1657014956808 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Van: Mark Millard Datum: maandag, 4 juli 2022 20:47 Aan: bob prohaska CC: Karl Denninger , freebsd-arm@freebsd.org Onderwerp: Re: 13.1R problems on Pi3 > > On 2022-Jul-4, at 11:25, bob prohaska wrote: > > > On Mon, Jul 04, 2022 at 12:17:15PM -0400, Karl Denninger wrote: > >> > >> On 7/4/2022 11:28, bob prohaska wrote: > >>> On Sun, Jul 03, 2022 at 10:36:35PM -0400, Karl Denninger wrote: > >>> > >>> Can any sense be made of the few ping responses obtained when ntp > >>> is coming up? It's looks as if something happens after ntp runs > >>> that blocks subsequent network traffic, but why starting an outbound > >>> ping should partly unblock things is obscure to me. > >> > >> Yes.?? The odds are reasonably high that there is confusion as to which MAC > >> address maps to which device.?? This implies there's a loop between the two > >> switches (e.g. there is more than one way for packets to get into and out of > >> each said switch to the other) or the two devices are claiming the same MAC > >> address and thus when each "speaks" and performs ARP it "grabs" the map > >> which works until the next one pipes up and it grabs it. > >> > > > > Looks like that's the problem. There's only one cable between switches, but > > here's what I get from ifconfig on each host: > > > > On the machine running 13.1-R attached to switch 2: > > bob@www:~ % ifconfig > > lo0: flags=8049 metric 0 mtu 16384 > > options=680003 > > inet6 ::1 prefixlen 128 > > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 > > inet 127.0.0.1 netmask 0xff000000 > > groups: lo > > nd6 options=21 > > ue0: flags=8843 metric 0 mtu 1500 > > options=80009 > >>>>>>>> ether b8:27:eb:71:46:4e > > inet 50.1.20.28 netmask 0xffffff00 broadcast 50.1.20.255 > > media: Ethernet autoselect (100baseTX ) > > status: active > > nd6 options=29 > > bob@www:~ % hostname > > www.zefox.org > > bob@www:~ % > > bob@www:~ % uname -a > > FreeBSD www.zefox.org 13.1-RELEASE FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC arm64 > > bob@www:~ % > > > > On the machine running an updated stable/13 system attached to switch 1: > > bob@pelorus:~ % ifconfig > > lo0: flags=8049 metric 0 mtu 16384 > > options=680003 > > inet6 ::1 prefixlen 128 > > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 > > inet 127.0.0.1 netmask 0xff000000 > > groups: lo > > nd6 options=21 > > ue0: flags=8843 metric 0 mtu 1500 > > options=80009 > >>>>>>> ether b8:27:eb:71:46:4e > > inet 50.1.20.24 netmask 0xffffff00 broadcast 50.1.20.255 > > media: Ethernet autoselect (100baseTX ) > > status: active > > nd6 options=29 > > bob@pelorus:~ % hostname > > pelorus.zefox.org > > bob@pelorus:~ % > > bob@pelorus:~ % uname -a > > FreeBSD pelorus.zefox.org 13.1-STABLE FreeBSD 13.1-STABLE #6 stable/13-n251601-2353343b324: Sun Jul 3 21:43:04 PDT 2022 bob@pelorus.zefox.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64 > > > > > > Thinking it over, I added the extra switch some time ago and didn't > > immediately notice any problems. Both Pi3s started out on the first > > switch (NetGear), with no obvdious problems. Later I probably moved > > one Pi3 to the second switch (D-Link) and started to notice troubles. > > Does this story make sense? > > > >> Each interface device from the factory is supposed to have a unique MAC > >> address.?? This can, for most interfaces, be overridden (modern Android > >> phones "randomize" it if told to as a "security" measure) but for obvious > >> reasons doing that can lead to problems. Collisions where multiple devices > >> are using the same MAC will lead to exactly the sort of thing you're seeing > >> because the switch is sending the packets to the wrong place. > >> > >> I've got a decent number of Pis of everything back to the "2" here and most > >> of the time several of them are on my network at once.?? I've not seen this > >> problem but I wouldn't exclude that both are claiming the same MAC and, if > >> so, that's what's causing the problem. > >> > > [example ifconfig output snipped] > >> > >> That MUST be unique on your LAN; the prefix (first three octets) is a vendor > >> code /*and the last three should never be duplicated by a vendor. */If you > >> are not setting it in /etc/rc.conf or elsewhere and there /are /duplicates > >> then a very bad thing happened when those units were manufactured -- set one > >> of them to something else. > >> > > > > Any pointers to MAC-setting methods appreciated..... > > My example is not the best fit because it is for DHCP > but in /etc/rc.conf I use (but showing "??"s): > > ifconfig_dwc0="ether ??:??:??:??:??:?? DHCP" > > to avoid its random assignment at power up. > > So for you I would guess: > > ifconfig_ue0="ether ??:??:??:??:??:?? inet 50.1.20.28 netmask 255.255.255.0" > > > === > Mark Millard > marklmi at yahoo.com > > > > > Hi, My Rpi3B+ does not have a random MAC on ue0. NB: It uses the muge driver: # devinfo | more ... bcm283x_dwcotg0 usbus1 uhub0 uhub1 uhub2 muge0 miibus0 ukphy0 ... Your current MAC is officially from the Raspberry Pi Foundation: https://hwaddress.com/oui-iab/B8-27-EB/ . Could you have hardcoded the MAC in a *.dtb file or other config in the /boot directory and copied that over to the other RPI? If you are going to assign some MAC yourself take a look at https://en.wikipedia.org/wiki/MAC_address#Universal_vs._local_(U/L_bit) to choose a locally administered MAC. Regards, Ronald. ------=_Part_134_2008716236.1657014956808 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit  

Van: Mark Millard <marklmi@yahoo.com>
Datum: maandag, 4 juli 2022 20:47
Aan: bob prohaska <fbsd@www.zefox.net>
CC: Karl Denninger <karl@denninger.net>, freebsd-arm@freebsd.org
Onderwerp: Re: 13.1R problems on Pi3

On 2022-Jul-4, at 11:25, bob prohaska <fbsd@www.zefox.net> wrote:

> On Mon, Jul 04, 2022 at 12:17:15PM -0400, Karl Denninger wrote:
>>
>> On 7/4/2022 11:28, bob prohaska wrote:
>>> On Sun, Jul 03, 2022 at 10:36:35PM -0400, Karl Denninger wrote:
>>>
>>> Can any sense be made of the few ping responses obtained when ntp
>>> is coming up? It's looks as if something happens after ntp runs
>>> that blocks subsequent network traffic, but why starting an outbound
>>> ping should partly unblock things is obscure to me.
>>
>> Yes.?? The odds are reasonably high that there is confusion as to which MAC
>> address maps to which device.?? This implies there's a loop between the two
>> switches (e.g. there is more than one way for packets to get into and out of
>> each said switch to the other) or the two devices are claiming the same MAC
>> address and thus when each "speaks" and performs ARP it "grabs" the map
>> which works until the next one pipes up and it grabs it.
>>
>
> Looks like that's the problem. There's only one cable between switches, but
> here's what I get from ifconfig on each host:
>
> On the machine running 13.1-R attached to switch 2:
> bob@www:~ % ifconfig
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
>   options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
>   inet6 ::1 prefixlen 128
>   inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
>   inet 127.0.0.1 netmask 0xff000000
>   groups: lo
>   nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
> ue0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>   options=80009<RXCSUM,VLAN_MTU,LINKSTATE>
>>>>>>>>    ether b8:27:eb:71:46:4e
>   inet 50.1.20.28 netmask 0xffffff00 broadcast 50.1.20.255
>   media: Ethernet autoselect (100baseTX <full-duplex>)
>   status: active
>   nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> bob@www:~ % hostname
> www.zefox.org
> bob@www:~ %
> bob@www:~ % uname -a
> FreeBSD www.zefox.org 13.1-RELEASE FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC arm64
> bob@www:~ %
>
> On the machine running an updated stable/13 system attached to switch 1:
> bob@pelorus:~ % ifconfig
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
>   options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
>   inet6 ::1 prefixlen 128
>   inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
>   inet 127.0.0.1 netmask 0xff000000
>   groups: lo
>   nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
> ue0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>   options=80009<RXCSUM,VLAN_MTU,LINKSTATE>
>>>>>>>     ether b8:27:eb:71:46:4e
>   inet 50.1.20.24 netmask 0xffffff00 broadcast 50.1.20.255
>   media: Ethernet autoselect (100baseTX <full-duplex>)
>   status: active
>   nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> bob@pelorus:~ % hostname
> pelorus.zefox.org
> bob@pelorus:~ %
> bob@pelorus:~ % uname -a
> FreeBSD pelorus.zefox.org 13.1-STABLE FreeBSD 13.1-STABLE #6 stable/13-n251601-2353343b324: Sun Jul  3 21:43:04 PDT 2022     bob@pelorus.zefox.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64
>
>
> Thinking it over, I added the extra switch some time ago and didn't
> immediately notice any problems. Both Pi3s started out on the first
> switch (NetGear), with no obvdious problems. Later I probably moved
> one Pi3 to the second switch (D-Link) and started to notice troubles.
> Does this story make sense?
>
>> Each interface device from the factory is supposed to have a unique MAC
>> address.?? This can, for most interfaces, be overridden (modern Android
>> phones "randomize" it if told to as a "security" measure) but for obvious
>> reasons doing that can lead to problems. Collisions where multiple devices
>> are using the same MAC will lead to exactly the sort of thing you're seeing
>> because the switch is sending the packets to the wrong place.
>>
>> I've got a decent number of Pis of everything back to the "2" here and most
>> of the time several of them are on my network at once.?? I've not seen this
>> problem but I wouldn't exclude that both are claiming the same MAC and, if
>> so, that's what's causing the problem.
>>
> [example ifconfig output snipped]
>>
>> That MUST be unique on your LAN; the prefix (first three octets) is a vendor
>> code /*and the last three should never be duplicated by a vendor. */If you
>> are not setting it in /etc/rc.conf or elsewhere and there /are /duplicates
>> then a very bad thing happened when those units were manufactured -- set one
>> of them to something else.
>>
>
> Any pointers to MAC-setting methods appreciated.....

My example is not the best fit because it is for DHCP
but in /etc/rc.conf I use (but showing "??"s):

ifconfig_dwc0="ether ??:??:??:??:??:?? DHCP"

to avoid its random assignment at power up.

So for you I would guess:

ifconfig_ue0="ether ??:??:??:??:??:?? inet 50.1.20.28 netmask 255.255.255.0"


===
Mark Millard
marklmi at yahoo.com

 


Hi,

My Rpi3B+ does not have a random MAC on ue0.

NB: It uses the muge driver:
# devinfo | more
...
      bcm283x_dwcotg0
        usbus1
          uhub0
            uhub1
              uhub2
                muge0
                  miibus0
                    ukphy0
...

Your current MAC is officially from the Raspberry Pi Foundation: https://hwaddress.com/oui-iab/B8-27-EB/ .

Could you have hardcoded the MAC in a *.dtb file or other config in the /boot directory and copied that over to the other RPI?

If you are going to assign some MAC yourself take a look at https://en.wikipedia.org/wiki/MAC_address#Universal_vs._local_(U/L_bit) to choose a locally administered MAC.

Regards,
Ronald.
  ------=_Part_134_2008716236.1657014956808--