Driver for Raspberry Pi 4 PCI-express controller

[Vincent Milum Jr]

My apologies, I forgot that my inbound and outbound email aliases were not
the same, so the FreeBSD-ARM mailing list rightfully rejected my email
before.

> hope it’s okay for you to keep this discussion public
Yes, it was intended to be a public discussion :)

> Did you test hot-unplugging/replugging the peripherals ?
Yes, for the most part it all went exactly as expected. Sometimes one of
the controllers would reset, other times it would work perfectly. A couple
times it would cause that endless error loop I reported before (in which
case, it seems only a power cycle solves it). Console log from an
unplug/replug event is listed below. This particular time is one of the
times the controller reset itself, and worked properly after.

> do you mean that not every boot-attempt does the same (for unknown reason)
Correct. Once or twice, I had the infinite controller resets on boot.

> or whether it`s caused by the pie-driver-electricity
This was my initial thought, too. I have been powering this Pi from the
USB-C port on my desktop workstation. Once or twice I saw the "low power"
warning on screen, so that is certainly a possibility. I've since had the
Pi running idle for over 24 hours without issue.

Additional testing...

USB to Serial adapter: Plugged in and /dev/cuaU0 properly showed up, but I
didn't hook it up to anything to test sending/receiving yet.
ugen0.8: <Prolific Technology Inc. USB-Serial Controller D> at usbus0
uplcom0 on uhub1
uplcom0: <Prolific Technology Inc. USB-Serial Controller D, class 0/0, rev
1.10/4.00, addr 7> on usbus0

Microsoft XBOX 360 Wired controller: Same as above. Plugged in, /dev/uhid0
showed up properly, but have not tested usage yet.
ugen0.8: <Microsoft Corporation Controller> at usbus0
uhid0 on uhub1
uhid0: <Microsoft Corporation Controller, rev 2.00/1.14, addr 7> on usbus0



unplug/replug USB hub with multiple devices attached, with "xhci_interrupt:
host system error" and controller reset:

ugen0.3: <Generic 4-Port USB 2.0 Hub> at usbus0 (disconnected)
uhub2: at uhub1, port 1, addr 2 (disconnected)
ugen0.4: <Microsoft Microsoft 5-Button Mouse with IntelliEye(TM)> at usbus0
(disconnected)
ums0: at uhub2, port 1, addr 3 (disconnected)
ums0: detached
ugen0.5: <Realtek USB 10/100/1000 LAN> at usbus0 (disconnected)
ure0: at uhub2, port 3, addr 4 (disconnected)
rgephy0: detached
miibus1: detached
ure0: detached
Jun  1 04:21:05 generic dhclient[7364]: receive_uhub2: detached
packet failed on ue0: Device not configured
Jun  1 04:21:05 generic dhclient[7364]: ioctl(SIOCGIFFLAGS) on ue0:
Operation not permitted
Jun  1 04:21:05 generic dhclient[7364]: Interface ue0 no longer appears
valid.
Jun  1 04:21:05 generic dhclient[7364]: No live interfaces to poll on -
exiting.
Jun  1 04:21:05 generic dhclient[7364]: exiting.
Jun  1 04:21:05 generic dhclient[7364]: connection closed
Jun  1 04:21:05 generic dhclient[7364]: exiting.
ugen0.3: <Generic 4-Port USB 2.0 Hub> at usbus0
uhub2 on uhub1
uhub2: <Generic 4-Port USB 2.0 Hub, class 9/0, rev 2.10/1.04, addr 2> on
usbus0
uhub2: MTT enabled
uhub2: 4 ports with 4 removable, self powered
ugen0.4: <Microsoft Microsoft 5-Button Mouse with IntelliEye(TM)> at usbus0
ums0 on uhub2
ums0: <Microsoft Microsoft 5-Button Mouse with IntelliEye(TM), class 0/0,
rev 1.10/3.00, addr 3> on usbus0
ums0: 5 buttons and [XYZ] coordinates ID=0
ugen0.5: <Realtek USB 10/100/1000 LAN> at usbus0
ure0 on uhub2
ure0: <Realtek USB 10/100/1000 LAN, class 0/0, rev 2.10/30.00, addr 4> on
usbus0
miibus1: <MII bus> on ure0
rgephy0: <RTL8251/8153 1000BASE-T media interface> PHY 0 on miibus1
rgephy0:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX,
1000baseT-FDX, 1000baseT-FDX-master, auto
xhci_interrupt: host system error
xhci0: Resetting controller
uhub0: at usbus0, port 1, addr 1 (disconnected)
ugen0.2: <vendor 0x2109 USB2.0 Hub> at usbus0 (disconnected)
uhub1: at uhub0, port 1, addr 1 (disconnected)
ugen0.3: <Generic 4-Port USB 2.0 Hub> at usbus0 (disconnected)
uhub2: at uhub1, port 1, addr 2 (disconnected)
ugen0.4: <Microsoft Microsoft 5-Button Mouse with IntelliEye(TM)> at usbus0
(disconnected)
ums0: at uhub2, port 1, addr 3 (disconnected)
ums0: detached
ugen0.5: <Realtek USB 10/100/1000 LAN> at usbus0 (disconnected)
ure0: at uhub2, port 3, addr 4 (disconnected)
rgephy0: detached
miibus1: detached
Jun  1 04:21ure0: detached
:37 generic dhclient[8021]: receive_packet failed on ue0: Device not
configured
Jun  1 04:21:37 generic dhclient[8021]: ioctl(SIOCGIFFLAGS) on ue0:
Operation not permitted
Jun  1 04:21:37 generic dhclient[8021]: Interface ue0 no longer appears
valid.
Jun  1 04:21:37 generic dhclient[8021]: No live interfaces to poll on -
exiting.
Jun  1 04:21:37 generic dhclient[8021]: exiting.
Jun  1 04:21:37 generic dhclient[8021]: connection closed
Jun  1 04:21:37 generic dhclient[8021]: exiting.
uhub2: detached
uhub1: detached
ugen0.7: <Kingston DataTraveler 3.0> at usbus0 (disconnected)
umass0: at uhub0, port 3, addr 6 (disconnected)
da0 at umass-sim0 bus 0 scbus0 target 0 lun 0
da0: <Kingston DataTraveler 3.0 >  s/n E0D55EA573F5F3B1A9B21117 detached
(da0:umass-sim0:0:0:0): Periph destroyed
umass0: detached
uhub0: detached
xhci0: Controller halt timeout.
uhub0 on usbus0
uhub0: <0x1106 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub0: 5 ports with 4 removable, self powered
ugen0.2: <vendor 0x2109 USB2.0 Hub> at usbus0
uhub1 on uhub0
uhub1: <vendor 0x2109 USB2.0 Hub, class 9/0, rev 2.10/4.20, addr 1> on
usbus0
uhub1: 4 ports with 4 removable, self powered
ugen0.3: <Generic 4-Port USB 2.0 Hub> at usbus0
uhub2 on uhub1
uhub2: <Generic 4-Port USB 2.0 Hub, class 9/0, rev 2.10/1.04, addr 2> on
usbus0
uhub2: MTT enabled
uhub2: 4 ports with 4 removable, self powered
ugen0.4: <Microsoft Microsoft 5-Button Mouse with IntelliEye(TM)> at usbus0
ums0 on uhub2
ums0: <Microsoft Microsoft 5-Button Mouse with IntelliEye(TM), class 0/0,
rev 1.10/3.00, addr 3> on usbus0
ums0: 5 buttons and [XYZ] coordinates ID=0
ugen0.5: <Realtek USB 10/100/1000 LAN> at usbus0
ure0 on uhub2
ure0: <Realtek USB 10/100/1000 LAN, class 0/0, rev 2.10/30.00, addr 4> on
usbus0
miibus1: <MII bus> on ure0
rgephy0: <RTL8251/8153 1000BASE-T media interface> PHY 0 on miibus1
rgephy0:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX,
1000baseT-FDX, 1000baseT-FDX-master, auto
ugen0.6: <Kingston DataTraveler 3.0> at usbus0
umass0 on uhub0
umass0: <Kingston DataTraveler 3.0, class 0/0, rev 3.20/0.01, addr 5> on
usbus0
umass0:  SCSI over Bulk-Only; quirks = 0xc100
umass0:0:0: Attached to scbus0
da0 at umass-sim0 bus 0 scbus0 target 0 lun 0
da0: <Kingston DataTraveler 3.0 > Removable Direct Access SPC-4 SCSI device
da0: Serial Number E0D55EA573F5F3B1A9B21117
da0: 400.000MB/s transfers
da0: 14755MB (30218842 512 byte sectors)
da0: quirks=0x2<NO_6_BYTE>


On Sat, May 30, 2020 at 1:28 PM Klaus Küchemann <maciphone2 at googlemail.com>
wrote:

> Hi,
> thanks, Vincent, for your extensive testing, very helpful !
> `hope it’s okay for you to keep this discussion public(forward to
> freebsd-arm at freebsd.org) .
>
> > Am 30.05.2020 um 20:11 schrieb Vincent M. Jr <>:
> >
> > dmesg: https://dmesgd.nycbug.org/index.cgi?do=view&id=5523
> >
> > …….
> > It isn't perfectly stable, but at least initially working…..
>
> Yeah, initially (sometimes) works for me too...
> >
> > Testing it with a USB-C (with adapter) ethernet + 3-port hub. I'm using
> this for wired ethernet rather than the built in ethernet just for testing.
> >
> > On the three hub ports, I have a microsoft keyboard, microsoft mouse,
> and a kingston datatraveler thumbdriv
> > There is also a Logitech keyboard on one of the Pi's USB 2.0 ports
>
> Did you test hot-unplugging/replugging the peripherals ?
>
> >
> > For testing, I've created a single-drive ZFS pool with the kingston
> drive.
> >
> > I went to delete a test file, and got the following:
> >
> > xhci_interrupt: host system error
> > xhci0: Resetting controller
> > uhub0: at usbus0, port 1, addr 1 (disconnected)
> > ugen0.2: <vendor 0x2109 USB2.0 Hub> at usbus0 (disconnected)
> > uhub1: at uhub0, port 1, addr 1 (disconnected)
> > ugen0.3: <Generic 4-Port USB 2.0 Hub> at usbus0 (disconnected)
> > uhub2: at uhub1, port 1, addr 2 (disconnected)
> > ugen0.4: <Kingston DataTraveler 3.0> at usbus0 (disconnected)
> > umass0: at uhub2, port 1, addr 3 (disconnected)
> > (da0:umass-sim0:0:0:0): WRITE(10). CDB: 2a 00 00 a0 21 a5 00 00 80 00
> > (da0:umass-sim0:0:0:0): CAM status: CCB request completed with an error
> > (da0:umass-sim0:0:0:0): Retrying command, 3 more tries remain
> >
> > But every single device had to be re-detected which took ~5-10 seconds
> to complete.
>
> Yeah, experienced that too ...
>
> >
> >
> > On another boot attempt, I'm getting this repeatedly while at the login
> prompt:...usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT,
> ignored)...ugen0.2: <Unknown > at usbus0 (disconnected)….
>
> This is a  very interesting information   :
> <another boot attempt> … do you mean that not every boot-attempt does the
> same (for unknown reason)
> (That’s what I have experienced but I wasn’t sure whether  my uSD is
> damaged or whether it`s caused by the pie-driver-electricity ??
> Specially the <Unknown > at -thing only appears in some boot attempts(not
> all)
> In my tests the strange thing is that one boot-attempt succeeds and
> another not , while left the machine completely untouched between those
> attempts..
>
>
>
> > update: i've gotten this also while doing a lengthy transfer to USB ZFS
> pool, and the error cycling never stops.
>
>  I will try to make a ZFS-pool-test...
>
> >
> > xhci0: Controller reset timeout.
> > uhub0 on usbus0
> > uhub0: <0x1106 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
> > uhub0: 5 ports with 4 removable, self powered
> > xhci0: Resetting controller
> > usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
> > ugen0.2: <Unknown > at usbus0 (disconnected)
> > uhub_reattach_port: could not allocate new device
> > uhub0: at usbus0, port 1, addr 1 (disconnected)
> > uhub0: detached
> > xhci0: Controller halt timeout.
> > xhci0: Controller reset timeout.
> > xhci0: Controller reset timeout.
> > uhub0 on usbus0
> > uhub0: <0x1106 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
> > uhub0: 5 ports with 4 removable, self powered
> > xhci0: Resetting controller
> > usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
> > ugen0.2: <Unknown > at usbus0 (disconnected)
> > uhub_reattach_port: could not allocate new device
> > uhub0: at usbus0, port 1, addr 1 (disconnected)
> > uhub0: detached
> > xhci0: Controller halt timeout.
> > xhci0: Controller reset timeout.
> > xhci0: Controller reset timeout.
> >
> >
> > ……..
>
>  xhci0 is this :
>
> https://svnweb.freebsd.org/base/head/sys/dev/usb/controller/generic_xhci.c?view=markup
> (and the corresponding fdt/acpi-files)
>
> uhub0 is this :
> https://svnweb.freebsd.org/base/head/sys/dev/usb/usb_hub.c?view=markup
>
> Thanks again for testing,
> Regards
>
>
>
>
>
>
>