[Bug 260127] Kernel Panic enumerating PI7C9X2G304SV PCIe Switch on Ten64 board (NXP QorIQ LS1088A)

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260127

            Bug ID: 260127
           Summary: Kernel Panic enumerating PI7C9X2G304SV PCIe Switch on
                    Ten64 board (NXP QorIQ LS1088A)
           Product: Base System
           Version: CURRENT
          Hardware: arm64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: arm
          Assignee: freebsd-arm@FreeBSD.org
          Reporter: matt@traverse.com.au

Created attachment 229803
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=229803&action=edit
dmesg kernel panic with PI7C9X2G304SV enabled

Thanks to the recent addition of various NXP/Freescale QorIQ drivers,
particularly qoriq_dw_pci, as well as the upcoming DPAA2 driver by
dsl@mcusim.org, FreeBSD is very close to being usable on the Traverse
Technologies Ten64 board.

Unfortunately there appears to be an issue enumerating the Diodes (Pericom)
PI7C9X2G304SV PCIe switch. This is used on this board to create two PCIe slots
from one PCIe 2.0 lane (specifically, the MiniPCIe slots, see
https://ten64doc.traverse.com.au/hardware/architecture/)

The panic occurs even when there are no downstream devices connected to the
PI7C9X2G304SV.

When the PCIe controller that the PI7C9X2G304SV is connected to is disabled in
the FDT, FreeBSD is able to boot and use the PCIe controller without issue,
including using an NVMe SSD as the boot device.

Here is the PCIe topology from Linux:
# lspci -nn
0000:00:00.0 PCI bridge [0604]: Freescale Semiconductor Inc Device [1957:80c0]
(rev 10)
0001:00:00.0 PCI bridge [0604]: Freescale Semiconductor Inc Device [1957:80c0]
(rev 10)
0001:01:00.0 PCI bridge [0604]: Pericom Semiconductor Device [12d8:b304] (rev
01)
0001:02:01.0 PCI bridge [0604]: Pericom Semiconductor Device [12d8:b304] (rev
01)
0001:02:02.0 PCI bridge [0604]: Pericom Semiconductor Device [12d8:b304] (rev
01)
0002:00:00.0 PCI bridge [0604]: Freescale Semiconductor Inc Device [1957:80c0]
(rev 10)
0002:01:00.0 Non-Volatile memory controller [0108]: Silicon Motion, Inc.
SM2263EN/SM2263XT SSD Controller [126f:2263] (rev 03)

# lspci -t
-+-[0002:00]---00.0-[01-ff]----00.0
 +-[0001:00]---00.0-[01-ff]----00.0-[02-04]--+-01.0-[03]--
 |                                           \-02.0-[04]--
 \-[0000:00]---00.0-[01-ff]--

Note: The PCIe controller instance that the switch is connected to is at
0x3500000. The NVMe SSD is connected to pcie@3600000 (bus 0002:00:00.0 in
Linux), pcie@3400000 is connected to the M.2 Key B ("cellular" slot) which is
empty in this example.

By manipulating the FDT in U-Boot pcie@3500000 can be disabled allowing FreeBSD
to boot:

setenv setup_distroboot_efi 'sf read 0x80001000 0x580000 0x40000 && fsl_mc
lazyapply dpl 0x80001000; sf read $fdt_addr_r 0x600000 0x40000 && fdt addr
$fdt_addr_r && fdt resize 10000 && fdt boardsetup && fdt set "/soc/spi@20c0000"
"status" "disabled" && fdt set "/soc/pcie@3500000" "status" "disabled"'
saveenv
reboot

-- 
You are receiving this mail because:
You are the assignee for the bug.