Re: wake-on-lan lost from rpi4 (works on rpi3)

On 29 Oct 2022, at 0:16, Ronald Klop wrote:

> Van: Ronald Klop <ronald-lists@klop.ws>
> Datum: 25 oktober 2022 20:32
> Aan: mike@karels.net
> CC: freebsd-arm@freebsd.org
> Onderwerp: Re: wake-on-lan lost from rpi4 (works on rpi3)
>
>>
>>
>>
>> Van: Mike Karels Datum: dinsdag, 25 oktober 2022 18:11
>> Aan: Ronald Klop CC: freebsd-arm@freebsd.org
>> Onderwerp: Re: wake-on-lan lost from rpi4 (works on rpi3)
>>>
>>> I wrote:
>>>> On 25 Oct 2022, at 9:13, Ronald Klop wrote:
>>>
>>>>> Van: Mike Karels > > Datum: dinsdag, 25 oktober 2022 14:55
>>>>> Aan: Ronald Klop > > CC: freebsd-arm@freebsd.org
>>>>> Onderwerp: Re: wake-on-lan lost from rpi4 (works on rpi3)
>>>>>>
>>>>>> On 25 Oct 2022, at 6:57, Ronald Klop wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> When I do "wake ue0 `cat /home/ronald/freenas.ethernet `" on my RPI3B+ my NAS boots and tcpdump shows the following line on my rpi3 as well as on my router.
>>>>>>> 13:39:44.344111 IPX 00217046.6c:da:00:21:70:46.6cda > 00217046.6c:da:00:21:70:46.6cda: ipx-#6cda 65505
>>>>>>>
>>>>>>> When I do "wake genet0 `cat /home/ronald/freenas.ethernet `" on my RPI4 my NAS does not boot and tcpdump only show this on my rpi4 and *not* on my router.
>>>>>>> 13:37:26.448251 IPX 00217046.6c:da:00:21:70:46.6cda > 00217046.6c:da:00:21:70:46.6cda: ipx-#6cda 65505
>>>>>>>
>>>>>>> Firewall ipfw does not indicate that it blocks anything.
>>>>>>>
>>>>>>> RPI4 runs:
>>>>>>> FreeBSD rpi4 14.0-CURRENT FreeBSD 14.0-CURRENT #11 main-4f0c9b76cf: Sat Aug 13 23:59:19 CEST 2022     ronald@rpi4:/home/ronald/dev/obj/home/ronald/dev/freebsd/arm64.aarch64/sys/GENERIC-NODEBUG arm64
>>>>>>>
>>>>>>> genet0: flags=8943 metric 0 mtu 1500
>>>>>>>    options=68000b
>>>>>>>
>>>>>>>
>>>>>>> RPI3 runs:
>>>>>>> FreeBSD rpi3 13.1-RELEASE-p2 FreeBSD 13.1-RELEASE-p2 GENERIC arm64
>>>>>>>
>>>>>>> ue0: flags=8943 metric 0 mtu 1500
>>>>>>>    options=80009
>>>>>>>
>>>>>>>
>>>>>>> Does genet0 not support these packages?
>>>>>>> What can prevent this packet to go on the ethernet while tcpdump still shows it is outgoing on the interface?
>>>>>>> genet0 does have a vlan configured connected to a bridge0 for some jails
>>>>>>> vlan3: flags=8943 metric 0 mtu 1500
>>>>>>>    options=80000
>>>>>>>    groups: vlan
>>>>>>>    vlan: 3 vlanproto: 802.1q vlanpcp: 0 parent interface: genet0
>>>>>>>
>>>>>>> ue0 is itself connected to a bridge0
>>>>>>
>>>>>> Is the RPI3 also on a vlan and bridge?
>>>>>>
>>>>>> Is the router (or switch) expecting the packet on the vlan?
>>>>>> If so, maybe you need to send on vlan3 rather than genet0.
>>>>>>
>>>>>> The genet interface has some oddities about packet layouts;
>>>>>> that could be hitting here.  You could try disabling TCP TX
>>>>>> checksum offload: ifconfig genet0 -txcsum -txcsum6; that
>>>>>> simplifies some of the issues.
>>>>>>
>>>>>>         Mike
>>>>>>
>>>>>>> Regards,
>>>>>>> Ronald.
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> Thanks for the hint. I experimented some further.
>>>>>
>>>>> Disabling -txcsum -rxcsum didn't matter.
>>>
>>>> -rxcsum doesn't matter, but you need to do -txcsum6 as well as
>>>> -txcsum.  (See the code that calls gen_parse_tx.)
>>>
>>>>> But when I use bridge0 or vlan3 as device then it goes into the wire but on the VLAN which is not where my NAS is which I want to boot.
>>>>>
>>>>> Looking at the driver I found this interesting peace of code.
>>>>>
>>>>> static int
>>>>> gen_parse_tx(struct mbuf *m, int csum_flags) {
>>>>> ...
>>>>>        if (ether_type == ETHERTYPE_IP) {
>>>>>                COPY(((struct ip *)p)->ip_hl << 2);
>>>>>                offset += ((struct ip *)p)->ip_hl << 2;
>>>>>        } else if (ether_type == ETHERTYPE_IPV6) {
>>>>>                COPY(sizeof(struct ip6_hdr));
>>>>>                offset += sizeof(struct ip6_hdr);
>>>>>        } else {
>>>>>                /*
>>>>>                 * Unknown whether other cases require moving a header;
>>>>>                 * ARP works without.
>>>>>                 */
>>>>>        }
>>>>> ...
>>>>> }
>>>>>
>>>>> There is also some code which handles EHTERTYPE_VLAN.
>>>>>
>>>>> I don't have time to start debugging this today. But would this ring a bell to anybody in connection to a wake-on-lan packet?
>>>
>>>> I ran some experiments with tx checksum disabled, and it seems to
>>>> matter.  I have a change for the last block you listed that might
>>>> help, but I haven't tested it yet.
>>>
>>> This patch seems to work:
>>>
>>> diff --git a/sys/arm64/broadcom/genet/if_genet.c b/sys/arm64/broadcom/genet/if_genet.c
>>> index 327af8acbcdd..b7ab86e7931d 100644
>>> --- a/sys/arm64/broadcom/genet/if_genet.c
>>> +++ b/sys/arm64/broadcom/genet/if_genet.c
>>> @@ -1305,6 +1305,8 @@ gen_parse_tx(struct mbuf *m, int csum_flags)
>>>          * Unknown whether other cases require moving a header;
>>>          * ARP works without.
>>>          */
>>> +       COPY(min(gen_tx_hdr_min, m->m_len));
>>> +       offset += min(gen_tx_hdr_min, m->m_len);
>>>     }
>>>     return (offset);
>>>  #undef COPY
>>>
>>> It needs an updated comment, but should be testable.
>>>
>>>         Mike
>>>
>>>>> Regards,
>>>>> Ronald.
>>>
>>>
>>
>> Hi,
>>
>> Thanks. Disabling both -txcsum and -txcsum6 works for me also.
>> I can test your patch tommorow or on Thursday. I'm first testing a big mongodb 6.0 patch I hope to commit soon and this build occupies my rpi4 for some more hours.
>>
>> Regards,
>> Ronald.
>>
>
>
> Hi,
>
> Your patch works for me. I can send wake-on-lan packets now. Should I test other specific things also?

The patch affects anything that is not IPv4 or IPv6, including ARP
(which seems to work).  Anything else that writes via BPF would take
the same path.  I’m not thinking of much, though.

I’ll go ahead and commit this.

		Mike

> Regards,
>
> Ronald.