[Bug 264006] dwc: dwc interface is not able to send packets when changing the dwc interface's media from the auto-negotiation mode to the forced mode

From: <bugzilla-noreply_at_freebsd.org>
Date: Sun, 15 May 2022 21:36:28 UTC
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=264006

            Bug ID: 264006
           Summary: dwc: dwc interface is not able to send packets when
                    changing the dwc interface's media from the
                    auto-negotiation mode to the forced mode
           Product: Base System
           Version: CURRENT
          Hardware: arm64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: bin
          Assignee: bugs@FreeBSD.org
          Reporter: jiahali@blackberry.com

Created attachment 233945
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=233945&action=edit
Preliminary fix patch

Test Procedure

The dwc interface is directly connected to a host interface, "enp0s31f6" in my
test. The auto-negotiation is always enabled in the host interface. The dwc
interface will be initialized as autoselect mode by default.
The "flag0" option is used in the test to disable auto-negotiation and set the
forced mode

The test is running on the following release

root@generic:~ # uname -a
FreeBSD generic 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n254961-b91a48693a5:
Thu Apr 21 09:35:51 UTC 2022    
root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64

1. Check the config of dwc interface and the interface is able to ping the host
interface

root@generic:~ # ifconfig -m dwc0
dwc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
        capabilities=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
        ether fa:97:92:f6:f1:09
        inet 192.168.3.129 netmask 0xffffff00 broadcast 192.168.3.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        supported media:
                media autoselect instance 1
                media 1000baseT mediaopt full-duplex,master instance 1
                media 1000baseT mediaopt full-duplex instance 1
                media 100baseTX mediaopt full-duplex instance 1
                media 100baseTX instance 1
                media 10baseT/UTP mediaopt full-duplex instance 1
                media 10baseT/UTP instance 1
                media none instance 1
                media autoselect
                media 1000baseT mediaopt full-duplex,master
                media 1000baseT mediaopt full-duplex
                media 100baseTX mediaopt full-duplex
                media 100baseTX
                media 10baseT/UTP mediaopt full-duplex
                media 10baseT/UTP
                media none
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
root@generic:~ # ping -c 1 192.168.3.2
PING 192.168.3.2 (192.168.3.2): 56 data bytes
64 bytes from 192.168.3.2: icmp_seq=0 ttl=64 time=0.534 ms

--- 192.168.3.2 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.534/0.534/0.534/0.000 ms

2. Check the config and media status of the host interface, "enp0s31f6". The
auto-negotiation process is correct and completed.

$ ifconfig enp0s31f6
enp0s31f6: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.3.2  netmask 255.255.255.0  broadcast 192.168.3.255
        ether 8c:8c:aa:c1:2b:c3  txqueuelen 1000  (Ethernet)
        RX packets 4078  bytes 1205102 (1.2 MB)
        RX errors 0  dropped 52  overruns 0  frame 0
        TX packets 24612  bytes 3572527 (3.5 MB)
        TX errors 0  dropped 0 overruns 0  carrier 526  collisions 0
        device interrupt 16  memory 0xae380000-ae3a0000

$ sudo ethtool enp0s31f6
Settings for enp0s31f6:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: 1000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: off (auto)
        Supports Wake-on: pumbg
        Wake-on: d
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

3. Change the dwc interface from auto-negotiation mode to forced mode,
100baseTX mediaopt full-duplex,flag0.

root@generic:~ # ifconfig -m dwc0 media 100baseTX mediaopt full-duplex,flag0
root@generic:~ # ifconfig -m dwc0 
dwc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
        capabilities=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
        ether fa:97:92:f6:f1:09
        inet 192.168.3.129 netmask 0xffffff00 broadcast 192.168.3.255
        media: Ethernet 100baseTX <full-duplex>
        status: active
        supported media:
                media autoselect instance 1
                media 1000baseT mediaopt full-duplex,master instance 1
                media 1000baseT mediaopt full-duplex instance 1
                media 100baseTX mediaopt full-duplex instance 1
                media 100baseTX instance 1
                media 10baseT/UTP mediaopt full-duplex instance 1
                media 10baseT/UTP instance 1
                media none instance 1
                media autoselect
                media 1000baseT mediaopt full-duplex,master
                media 1000baseT mediaopt full-duplex
                media 100baseTX mediaopt full-duplex
                media 100baseTX
                media 10baseT/UTP mediaopt full-duplex
                media 10baseT/UTP
                media none
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

4. Check the media status of the host interface. The media set up between the
device with auto-negotiation enabled and the device with the forced mode is
correct and completed. The host interface is set at the same speed as the dwc
interface but its duplex mode is half.

$ sudo ethtool enp0s31f6
Settings for enp0s31f6:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: 100Mb/s
        Duplex: Half
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: off (auto)
        Supports Wake-on: pumbg
        Wake-on: d
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

5. The dwc interface is not able to ping the host interface anymore. Based on
the result of "netstat", the packet is sent from the network layer but not
recorded at the link-layer level

root@generic:~ # netstat -nI dwc0
Name    Mtu Network       Address              Ipkts Ierrs Idrop    Opkts Oerrs
 Coll
dwc0   1500 <Link#1>      fa:97:92:f6:f1:09      130     0     0      132     0
    0
dwc0      - 192.168.3.0/2 192.168.3.129           54     -     -        6     -
    -
root@generic:~ # ping -c 1 192.168.3.2
PING 192.168.3.2 (192.168.3.2): 56 data bytes

--- 192.168.3.2 ping statistics ---
1 packets transmitted, 0 packets received, 100.0% packet loss
root@generic:~ # netstat -nI dwc0
Name    Mtu Network       Address              Ipkts Ierrs Idrop    Opkts Oerrs
 Coll
dwc0   1500 <Link#1>      fa:97:92:f6:f1:09      130     0     0      132     0
    0
dwc0      - 192.168.3.0/2 192.168.3.129           54     -     -        7     -
    -

6. Set the dwc interface into another forced mode. "PING" is also failed.

root@generic:~ # ifconfig -m dwc0 media 10baseT/UTP mediaopt flag0

root@generic:~ # ifconfig -m dwc0
dwc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
        capabilities=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
        ether fa:97:92:f6:f1:09
        inet 192.168.3.129 netmask 0xffffff00 broadcast 192.168.3.255
        media: Ethernet 10baseT/UTP (10baseT/UTP <half-duplex>)
        status: active
        supported media:
                media autoselect instance 1
                media 1000baseT mediaopt full-duplex,master instance 1
                media 1000baseT mediaopt full-duplex instance 1
                media 100baseTX mediaopt full-duplex instance 1
                media 100baseTX instance 1
                media 10baseT/UTP mediaopt full-duplex instance 1
                media 10baseT/UTP instance 1
                media none instance 1
                media autoselect
                media 1000baseT mediaopt full-duplex,master
                media 1000baseT mediaopt full-duplex
                media 100baseTX mediaopt full-duplex
                media 100baseTX
                media 10baseT/UTP mediaopt full-duplex
                media 10baseT/UTP
                media none
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
root@generic:~ # netstat -nI dwc0
Name    Mtu Network       Address              Ipkts Ierrs Idrop    Opkts Oerrs
 Coll
dwc0   1500 <Link#1>      fa:97:92:f6:f1:09      134     0     0      132     0
    0
dwc0      - 192.168.3.0/2 192.168.3.129           56     -     -        7     -
    -
root@generic:~ # ping -c 1 192.168.3.2
PING 192.168.3.2 (192.168.3.2): 56 data bytes

--- 192.168.3.2 ping statistics ---
1 packets transmitted, 0 packets received, 100.0% packet loss
root@generic:~ # netstat -nI dwc0
Name    Mtu Network       Address              Ipkts Ierrs Idrop    Opkts Oerrs
 Coll
dwc0   1500 <Link#1>      fa:97:92:f6:f1:09      134     0     0      132     0
    0
dwc0      - 192.168.3.0/2 192.168.3.129           56     -     -        8     -
    -

7. Check the media status of the host interface. The media set up between the
device with auto-negotiation enabled and the device with the forced mode is
correct and completed.

$ sudo ethtool enp0s31f6
Settings for enp0s31f6:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: 10Mb/s
        Duplex: Half
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: on (auto)
        Supports Wake-on: pumbg
        Wake-on: d
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

8. Change the dwc interface from forced to auto-select mode. "PING" is
successful.

root@generic:~ # ifconfig -m dwc0 media autoselect
root@generic:~ # ifconfig -m dwc0 
dwc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
        capabilities=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
        ether fa:97:92:f6:f1:09
        inet 192.168.3.129 netmask 0xffffff00 broadcast 192.168.3.255
        media: Ethernet autoselect (1000baseT <full-duplex,master>)
        status: active
        supported media:
                media autoselect instance 1
                media 1000baseT mediaopt full-duplex,master instance 1
                media 1000baseT mediaopt full-duplex instance 1
                media 100baseTX mediaopt full-duplex instance 1
                media 100baseTX instance 1
                media 10baseT/UTP mediaopt full-duplex instance 1
                media 10baseT/UTP instance 1
                media none instance 1
                media autoselect
                media 1000baseT mediaopt full-duplex,master
                media 1000baseT mediaopt full-duplex
                media 100baseTX mediaopt full-duplex
                media 100baseTX
                media 10baseT/UTP mediaopt full-duplex
                media 10baseT/UTP
                media none
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
root@generic:~ # ping -c 1 192.168.3.2
PING 192.168.3.2 (192.168.3.2): 56 data bytes
64 bytes from 192.168.3.2: icmp_seq=0 ttl=64 time=0.302 ms

--- 192.168.3.2 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.302/0.302/0.302/0.000 ms

9. Check the media status of the host interface, "enp0s31f6". The
auto-negotiation process is correct and completed.

$ sudo ethtool enp0s31f6
Settings for enp0s31f6:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: 1000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: off (auto)
        Supports Wake-on: pumbg
        Wake-on: d
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

Analysis

I tried to step through the code about setting the media type into a forced
mode and attach a patch for a quick fix to the problem in my personal
development environment.

My development environment is based on FreeBSD 13.0-RELEASE-p11. And I do not
have a development environment based on the current release of Freebsd image.

When setting the interface into a forced mode, the "mii_media_status" will be
set to "DOWN" firstly and the "mii_media_active" will be set as the
corresponding forced mode. The mii_physubr.c/mii_phy_update() function will
invoke the if_dwc.c/dwc_miibus_statchg() function to set up the "link_is_up"
field in "dwc_softc" struct as the "mii_media_status". If the "link_is_up" is
not set as "UP", the interface is not able to send any packets. However, the
dwc_miibus_statchg() will only be invoked when the "mii_media_active" is
changed or mii.c/mii_mediachg() function is invoked. The change of
"mii_media_status" will invoke the if.c/if_link_state_change() function to
change the state in the dwc's "ifnet" struct. 

The "mii_media_status" has to wait for a while to let the PHY hardware setting
be completed. Then, the "mii_media_status" will be set as "UP" in the dwc
interface's callout function, dwc_tick() when the hardware setting is
completed. But the "mii_media_active" is not changed and the
if_dwc.c/dwc_miibus_statchg() function won't be invoked anymore. The
"dwc_softc" struct's "link_is_up" field is still "down". Therefore, the packets
sending is failed.

-- 
You are receiving this mail because:
You are the assignee for the bug.