Re: FYI: Rock64 USB3 port no longer works for main [so: 14] (looks like dtb changes invalidating use of the old .dtbo and needing kernel changes)

From: Mark Millard <marklmi_at_yahoo.com>
Date: Tue, 15 Nov 2022 23:06:24 UTC
On Nov 15, 2022, at 13:12, Emmanuel Vadot <manu@bidouilliste.com> wrote:

> On Tue, 15 Nov 2022 13:03:45 -0800
> Mark Millard <marklmi@yahoo.com> wrote:
> 
>> On Nov 15, 2022, at 05:05, Emmanuel Vadot <manu@bidouilliste.com> wrote:
>> 
>>> On Fri, 4 Nov 2022 12:31:51 -0700
>>> Mark Millard <marklmi@yahoo.com> wrote:
>>> 
>>>> On 2022-Oct-22, at 23:00, Mark Millard <marklmi@yahoo.com> wrote:
>>>> 
>>>>> Well, turns out that part of the "Import device-tree files
>>>>> from Linux 5.14" is:
>>>>> 
>>>>> https://cgit.freebsd.org/src/commit/sys/contrib/device-tree/src/arm64/rockchip/rk3328-rock64.dts?id=5956d97f4b32
>>>>> 
>>>>> which has:
>>>>> . . .
>>>> 
>>>> 
>>> 
>>> Hi Mark,
>>> 
>>> See https://reviews.freebsd.org/D37392 (and child reviews) for a fix.
>>> This was indeed the import of the new DTS files that caused the first
>>> problem (there is no glue node in rk3328.dtsi like in other SoCs or
>>> like our overlay). The other commit responsible for breaking USB3
>>> support was the addition to RK356x SoC, the check was bad for when to
>>> force USB2.
>> 
>> Thanks.
>> 
>> I applied the diff and the 2 child diff's and rebuilt and
>> installed, including updating the kernel on the e.MMC that
>> is historically used to mount the rootfs on USB3 when the
>> USB3 drive is plugged in there. (U-boot does not handle
>> the USB context I want.)
>> 
>> . . .
>> 
>> 
> 
> Looks like you're missing https://reviews.freebsd.org/D37394

Hopefully this time I have all the childern, grandchildern, etc.
Sorry for the screwup.

But the behavior observed is unchanged in this attempt.

A diff of the console output of booting via USB2 this time
vs. the prior report's USB3 failure log reveals no differences
between the latter part of the block of:

clknode_link_recalc: Attempt to use unresolved linked clock: hdmi_phy
Cannot get frequency for clk: hdmi_phy, error: 9

and the later:

Trying to mount root from ufs:/dev/gpt/Rock64root []...
uhub2: 1 port with 1 removable, self powered
uhub1: 1 port with 1 removable, self powered
uhub0: 1 port with 1 removable, self powered

No DWC3, XHCI, etc.

But a boot -v does report:

ofwbus0: <usb@ff600000> mem 0xff600000-0xff6fffff irq 50 compat rockchip,rk3328-xhci (no driver attached)

Note the "xhci" instead of "dwc3".

The overall list of no driver notices that do not mention "disabled" is
listed below. It does reference a "rockchip,rk3328-usb2phy" as well as
the above.

ofwbus0: <opp_table0> compat operating-points-v2 (no driver attached)
ofwbus0: <display-subsystem> compat rockchip,display-subsystem (no driver attached)
ofwbus0: <spdif@ff030000> mem 0xff030000-0xff030fff irq 11 compat rockchip,rk3328-spdif (no driver attached)
ofwbus0: <pdm@ff040000> mem 0xff040000-0xff040fff disabled compat rockchip,pdm (no driver attached)
rk_grf0: <io-domains> mem 0xff100000-0xff100fff compat rockchip,rk3328-io-voltage-domain (no driver attached)
rk_grf0: <gpio> mem 0xff100000-0xff100fff compat rockchip,rk3328-grf-gpio (no driver attached)
rk_grf0: <power-controller> mem 0xff100000-0xff100fff compat rockchip,rk3328-power-controller (no driver attached)
rk_grf0: <reboot-mode> mem 0xff100000-0xff100fff compat syscon-reboot-mode (no driver attached)
ofwbus0: <watchdog@ff1a0000> mem 0xff1a0000-0xff1a00ff irq 20 compat rockchip,rk3328-wdt (no driver attached)
ofwbus0: <dmac@ff1f0000> mem 0xff1f0000-0xff1f3fff irq 22,23 compat arm,pl330 (no driver attached)
ofwbus0: <efuse@ff260000> mem 0xff260000-0xff26004f compat rockchip,rk3328-efuse (no driver attached)
ofwbus0: <gpu@ff300000> mem 0xff300000-0xff33ffff irq 26,27,28,29,30,31,32 compat rockchip,rk3328-mali (no driver attached)
ofwbus0: <video-codec@ff350000> mem 0xff350000-0xff3507ff irq 35 compat rockchip,rk3328-vpu (no driver attached)
ofwbus0: <iommu@ff350800> mem 0xff350800-0xff35083f irq 36 compat rockchip,iommu (no driver attached)
ofwbus0: <vop@ff370000> mem 0xff370000-0xff373efb irq 38 compat rockchip,rk3328-vop (no driver attached)
ofwbus0: <iommu@ff373f00> mem 0xff373f00-0xff373fff irq 39 compat rockchip,iommu (no driver attached)
ofwbus0: <hdmi@ff3c0000> mem 0xff3c0000-0xff3dffff irq 40,41 compat rockchip,rk3328-dw-hdmi (no driver attached)
ofwbus0: <codec@ff410000> mem 0xff410000-0xff410fff compat rockchip,rk3328-codec (no driver attached)
ofwbus0: <phy@ff430000> mem 0xff430000-0xff43ffff irq 42 compat rockchip,rk3328-hdmi-phy (no driver attached)
simple_mfd0: <usb2phy@100> mem 0-0xff44ffff,0-0xffff compat rockchip,rk3328-usb2phy (no driver attached)
ofwbus0: <usb@ff600000> mem 0xff600000-0xff6fffff irq 50 compat rockchip,rk3328-xhci (no driver attached)
ofwbus0: <ir-receiver> compat gpio-ir-receiver (no driver attached)
ofwbus0: <spdif-dit> compat linux,spdif-dit (no driver attached)
ofwbus0: <dmc> mem 0xff400000-0xff400fff,0xff780000-0xff782fff,0xff100000-0xff100fff,0xff440000-0xff440fff,0xff720000-0xff720fff,0xff798000-0xff798fff compat rockchip,rk3328-dmc (no driver attached)
ofwbus0: <smbios> compat u-boot,sysinfo-smbios (no driver attached)
pcm0: no driver attached to codec node
pcm1: no driver attached to codec node
pcm2: no driver attached to cpu node



For reference:

# git -C /usr/main-src/ diff sys/dev/usb/controller/ sys/arm64/rockchip/ sys/dts/ sys/modules/
diff --git a/sys/arm64/rockchip/rk_dwc3.c b/sys/arm64/rockchip/rk_dwc3.c
index 8582f7a86999..645a1cffbd95 100644
--- a/sys/arm64/rockchip/rk_dwc3.c
+++ b/sys/arm64/rockchip/rk_dwc3.c
@@ -54,12 +54,10 @@ __FBSDID("$FreeBSD$");
 #include <dev/extres/syscon/syscon.h>
 
 enum rk_dwc3_type {
-       RK3328 = 1,
-       RK3399,
+       RK3399 = 1,
 };
 
 static struct ofw_compat_data compat_data[] = {
-       { "rockchip,rk3328-dwc3",       RK3328 },
        { "rockchip,rk3399-dwc3",       RK3399 },
        { NULL,                         0 }
 };
diff --git a/sys/dev/usb/controller/dwc3.c b/sys/dev/usb/controller/dwc3.c
index 2e8f868bc47b..d5e3b3f50a9d 100644
--- a/sys/dev/usb/controller/dwc3.c
+++ b/sys/dev/usb/controller/dwc3.c
@@ -86,6 +86,14 @@ struct snps_dwc3_softc {
        bus_space_tag_t         bst;
        bus_space_handle_t      bsh;
        uint32_t                snpsid;
+       uint32_t                snpsversion;
+       uint32_t                snpsrevision;
+       uint32_t                snpsversion_type;
+#ifdef FDT
+       clk_t                   clk_ref;
+       clk_t                   clk_suspend;
+       clk_t                   clk_bus;
+#endif
 };
 
 #define        DWC3_WRITE(_sc, _off, _val)             \
@@ -384,8 +392,31 @@ snps_dwc3_common_attach(device_t dev, bool is_fdt)
        sc->bsh = rman_get_bushandle(sc->mem_res);
 
        sc->snpsid = DWC3_READ(sc, DWC3_GSNPSID);
-       if (bootverbose)
-               device_printf(sc->dev, "snps id: %#012x\n", sc->snpsid);
+       sc->snpsversion = DWC3_VERSION(sc->snpsid);
+       sc->snpsrevision = DWC3_REVISION(sc->snpsid);
+       if (sc->snpsversion == DWC3_1_IP_ID ||
+           sc->snpsversion == DWC3_2_IP_ID) {
+               sc->snpsrevision = DWC3_READ(sc, DWC3_1_VER_NUMBER);
+               sc->snpsversion_type = DWC3_READ(sc, DWC3_1_VER_TYPE);
+       }
+       if (bootverbose) {
+               switch (sc->snpsversion) {
+               case DWC3_IP_ID:
+                       device_printf(sc->dev, "SNPS Version: DWC3 (%x %x)\n",
+                           sc->snpsversion, sc->snpsrevision);
+                       break;
+               case DWC3_1_IP_ID:
+                       device_printf(sc->dev, "SNPS Version: DWC3.1 (%x %x %x)\n",
+                           sc->snpsversion, sc->snpsrevision,
+                           sc->snpsversion_type);
+                       break;
+               case DWC3_2_IP_ID:
+                       device_printf(sc->dev, "SNPS Version: DWC3.2 (%x %x %x)\n",
+                           sc->snpsversion, sc->snpsrevision,
+                           sc->snpsversion_type);
+                       break;
+               }
+       }
 #ifdef DWC3_DEBUG
        snps_dwc3_dump_ctrlparams(sc);
 #endif
@@ -394,9 +425,32 @@ snps_dwc3_common_attach(device_t dev, bool is_fdt)
        if (!is_fdt)
                goto skip_phys;
 
-       /* Get the phys */
        node = ofw_bus_get_node(dev);
 
+       /* Get the clocks if any */
+       if (ofw_bus_is_compatible(dev, "rockchip,rk3328-dwc3") == 1) {
+               if (clk_get_by_ofw_name(dev, node, "ref_clk", &sc->clk_ref) != 0)
+                       device_printf(dev, "Cannot get ref_clk\n");
+               if (clk_get_by_ofw_name(dev, node, "suspend_clk", &sc->clk_suspend) != 0)
+                       device_printf(dev, "Cannot get suspend_clk\n");
+               if (clk_get_by_ofw_name(dev, node, "bus_clk", &sc->clk_bus) != 0)
+                       device_printf(dev, "Cannot get bus_clk\n");
+       }
+
+       if (sc->clk_ref != NULL) {
+               if (clk_enable(sc->clk_ref) != 0)
+                       device_printf(dev, "Cannot enable ref_clk\n");
+       }
+       if (sc->clk_suspend != NULL) {
+               if (clk_enable(sc->clk_suspend) != 0)
+                       device_printf(dev, "Cannot enable suspend_clk\n");
+       }
+       if (sc->clk_bus != NULL) {
+               if (clk_enable(sc->clk_bus) != 0)
+                       device_printf(dev, "Cannot enable bus_clk\n");
+       }
+
+       /* Get the phys */
        usb2_phy = usb3_phy = NULL;
        error = phy_get_by_ofw_name(dev, node, "usb2-phy", &usb2_phy);
        if (error == 0 && usb2_phy != NULL)
@@ -404,12 +458,19 @@ snps_dwc3_common_attach(device_t dev, bool is_fdt)
        error = phy_get_by_ofw_name(dev, node, "usb3-phy", &usb3_phy);
        if (error == 0 && usb3_phy != NULL)
                phy_enable(usb3_phy);
-       else {
-               reg = DWC3_READ(sc, DWC3_GUCTL1);
-               if (bootverbose)
-                       device_printf(dev, "Forcing USB2 clock only\n");
-               reg |= DWC3_GUCTL1_DEV_FORCE_20_CLK_FOR_30_CLK;
-               DWC3_WRITE(sc, DWC3_GUCTL1, reg);
+       if (sc->snpsversion == DWC3_IP_ID) {
+               if (sc->snpsrevision >= 0x290A) {
+                       uint32_t hwparams3;
+
+                       hwparams3 = DWC3_READ(sc, DWC3_GHWPARAMS3);
+                       if (DWC3_HWPARAMS3_SSPHY(hwparams3) == DWC3_HWPARAMS3_SSPHY_DISABLE) {
+                               reg = DWC3_READ(sc, DWC3_GUCTL1);
+                               if (bootverbose)
+                                       device_printf(dev, "Forcing USB2 clock only\n");
+                               reg |= DWC3_GUCTL1_DEV_FORCE_20_CLK_FOR_30_CLK;
+                               DWC3_WRITE(sc, DWC3_GUCTL1, reg);
+                       }
+               }
        }
        snps_dwc3_configure_phy(sc, node);
 skip_phys:
@@ -427,6 +488,16 @@ snps_dwc3_common_attach(device_t dev, bool is_fdt)
        snsp_dwc3_dump_regs(sc, "Post XHCI init");
 #endif
 
+#ifdef FDT
+       if (error) {
+               if (sc->clk_ref != NULL)
+                       clk_disable(sc->clk_ref);
+               if (sc->clk_suspend != NULL)
+                       clk_disable(sc->clk_suspend);
+               if (sc->clk_bus != NULL)
+                       clk_disable(sc->clk_bus);
+       }
+#endif
        return (error);
 }
 
diff --git a/sys/dev/usb/controller/dwc3.h b/sys/dev/usb/controller/dwc3.h
index 21a87a1917ee..fd61d1129df3 100644
--- a/sys/dev/usb/controller/dwc3.h
+++ b/sys/dev/usb/controller/dwc3.h
@@ -31,6 +31,15 @@
 #ifndef _DWC3_H_
 #define        _DWC3_H_
 
+#define        DWC3_IP_ID              0x5533
+#define        DWC3_1_IP_ID            0x3331
+#define        DWC3_2_IP_ID            0x3332
+
+#define        DWC3_VERSION_MASK       0xFFFF0000
+#define        DWC3_REVISION_MASK      0xFFFF
+#define        DWC3_VERSION(x)         (((x) & DWC3_VERSION_MASK) >> 16)
+#define        DWC3_REVISION(x)        ((x) & DWC3_REVISION_MASK)
+
 #define        DWC3_GSBUSCFG0          0xc100
 #define        DWC3_GSBUSCFG1          0xc104
 #define        DWC3_GTXTHRCFG          0xc108
@@ -80,6 +89,9 @@
 #define        DWC3_GPRTBIMAP_HSLO     0xc180
 #define        DWC3_GPRTBIMAP_FSLO     0xc188
 
+#define        DWC3_1_VER_NUMBER       0xc1a0
+#define        DWC3_1_VER_TYPE         0xc1a4
+
 #define        DWC3_GUSB2PHYCFG0       0xc200
 #define         DWC3_GUSB2PHYCFG0_PHYSOFTRST           (1 << 31)
 #define         DWC3_GUSB2PHYCFG0_U2_FREECLK_EXISTS    (1 << 30)
@@ -98,6 +110,11 @@
 #define         DWC3_GUSB3PIPECTL0_DELAYP1TRANS        (1 << 18)
 #define         DWC3_GUSB3PIPECTL0_SUSPENDUSB3         (1 << 17)
 
+#define        DWC3_HWPARAMS3_SSPHY(x)                 (x & 0x3)
+#define         DWC3_HWPARAMS3_SSPHY_DISABLE           0
+#define         DWC3_HWPARAMS3_SSPHY_GEN1              1
+#define         DWC3_HWPARAMS3_SSPHY_GEN2              2
+
 #define        DWC3_GTXFIFOSIZ(x)      (0xc300 + 0x4 * (x))
 #define        DWC3_GRXFIFOSIZ(x)      (0xc380 + 0x4 * (x))
 #define        DWC3_GEVNTADRLO0                0xc400
diff --git a/sys/dts/arm64/overlays/rk3328-dwc3.dtso b/sys/dts/arm64/overlays/rk3328-dwc3.dtso
deleted file mode 100644
index bade60b60d57..000000000000
--- a/sys/dts/arm64/overlays/rk3328-dwc3.dtso
+++ /dev/null
@@ -1,39 +0,0 @@
-/dts-v1/;
-/plugin/;
-
-#include <dt-bindings/clock/rk3328-cru.h>
-#include <dt-bindings/interrupt-controller/arm-gic.h>
-#include <dt-bindings/interrupt-controller/irq.h>
-
-/ {
-       compatible = "rockchip,rk3328";
-};
-
-&{/} {
-       usbdrd3: usb@ff600000 {
-               compatible = "rockchip,rk3328-dwc3";
-               clocks = <&cru SCLK_USB3OTG_REF>, <&cru SCLK_USB3OTG_SUSPEND>,
-                        <&cru ACLK_USB3OTG>;
-               clock-names = "ref_clk", "suspend_clk",
-                             "bus_clk";
-               #address-cells = <2>;
-               #size-cells = <2>;
-               ranges;
-               status = "okay";
-
-               usbdrd_dwc3: dwc3@ff600000 {
-                       compatible = "snps,dwc3";
-                       reg = <0x0 0xff600000 0x0 0x100000>;
-                       interrupts = <GIC_SPI 67 IRQ_TYPE_LEVEL_HIGH>;
-                       dr_mode = "host";
-                       phy_type = "utmi_wide";
-                       snps,dis_enblslpm_quirk;
-                       snps,dis-u2-freeclk-exists-quirk;
-                       snps,dis_u2_susphy_quirk;
-                       snps,dis_u3_susphy_quirk;
-                       snps,dis-del-phy-power-chg-quirk;
-                       snps,dis-tx-ipgap-linecheck-quirk;
-                       status = "okay";
-               };
-       };
-};
diff --git a/sys/modules/dtb/rockchip/Makefile b/sys/modules/dtb/rockchip/Makefile
index 1e0c6a2d4362..4f4a21f51a39 100644
--- a/sys/modules/dtb/rockchip/Makefile
+++ b/sys/modules/dtb/rockchip/Makefile
@@ -22,7 +22,6 @@ DTS=  \
 DTSO=  rk3328-analog-sound.dtso \
        rk3328-i2c0.dtso \
        rk3328-uart1.dtso \
-       rk3328-dwc3.dtso \
        rk3399-mmc0-disable.dtso \
        rk3399-mmc1-disable.dtso \
        rk3399-sdhci-disable.dtso

===
Mark Millard
marklmi at yahoo.com