debugging a msi interrupt panic
- Reply: John Hay : "Re: debugging a msi interrupt panic"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Fri, 22 Sep 2023 11:00:56 UTC
Hi, How do one debug msi interrupts on FreeBSD when writing a device driver? I'm writing a driver for a OCP TimeCard. The card has several different functions, each "tied" to a specific msi interrupt. So it would be useful to use multiple msi interrupts. The PCIe functionality is a Xilinx AXI PCIe blob, with 32 MSI interrupts. It does not have MSI-X capability. <snip> # pciconf -lbc none1@pci0:1:0:0: none0@pci0:1:0:0: class=0x058000 rev=0x00 hdr=0x00 vendor=0x1d9b device=0x0400 subvendor=0x10ee subdevice=0x0007 bar [10] = type Memory, range 32, base 0xf4000000, size 33554432, enabled cap 01[40] = powerspec 3 supports D0 D3 current D0 cap 05[48] = MSI supports 32 messages, 64 bit cap 10[60] = PCI-Express 2 endpoint max data 256(256) NS max read 512 link x1(x1) speed 2.5(2.5) ASPM disabled(L0s) ecap 0003[100] = Serial 1 0000000000000000 </snip> Currently in attach, I'm doing the following (error checking removed): <snip> pci_enable_busmaster(dev); /* msi irq needs it enabled, at least for the AXI core */ sc->sc_msi = pci_msi_count(dev); pci_alloc_msi(dev, &sc->sc_msi) loop for each function /* currently just a single iteration to get one function working */ port->pp_irid = port->intr_num + 1; /* msi irq no starts at one */ port->pp_ires = bus_alloc_resource_any(dev, SYS_RES_IRQ, &port->pp_irid, RF_ACTIVE | RF_SHAREABLE); bus_setup_intr(dev, port->pp_ires, INTR_TYPE_TTY | INTR_MPSAFE, timecard_intr, NULL, port, &port->p_ihandle) </snip> I did define a variable timecard_intr_count that timecard_intr() should increment, but it does not change, so timecard_intr() probably does not get called. Here is the console output of the panic and a show apic and show lapic. The driver is not compiled in, but loaded later manually. <snip> pci0: driver added found-> vendor=0x8086, dev=0x1e3a, revid=0x04 domain=0, bus=0, slot=22, func=0 class=07-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=16 powerspec 3 supports D0 D3 current D0 MSI supports 1 message, 64 bit pci0:0:22:0: reprobing on driver added timecard0: TimeCard Probe Vendor ID : 0x8086 Device ID : 0x1e3a pci1: driver added found-> vendor=0x1d9b, dev=0x0400, revid=0x00 domain=0, bus=1, slot=0, func=0 class=05-80-00, hdrtype=0x00, mfdev=0 cmdreg=0x0002, statreg=0x0010, cachelnsz=16 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) powerspec 3 supports D0 D3 current D0 MSI supports 32 messages, 64 bit pci0:1:0:0: reprobing on driver added timecard0: TimeCard Probe Vendor ID : 0x1d9b Device ID : 0x400 We've got the TimeCard, probe successful! timecard0: TimeCard Probe Vendor ID : 0x1d9b Device ID : 0x400 We've got the TimeCard, probe successful! timecard0: <TimeCard> mem 0xf4000000-0xf5ffffff at device 0.0 on pci1 TimeCard Attach for : deviceID : 0x4001d9b Address of timecard_intr_count 0xffffffff8314e400 timecard0: attempting to allocate 32 MSI vectors (32 supported) msi: routing MSI IRQ 38 to local APIC 4 vector 80 msi: routing MSI IRQ 39 to local APIC 4 vector 81 msi: routing MSI IRQ 40 to local APIC 4 vector 82 msi: routing MSI IRQ 41 to local APIC 4 vector 83 msi: routing MSI IRQ 42 to local APIC 4 vector 84 msi: routing MSI IRQ 43 to local APIC 4 vector 85 msi: routing MSI IRQ 44 to local APIC 4 vector 86 msi: routing MSI IRQ 45 to local APIC 4 vector 87 msi: routing MSI IRQ 46 to local APIC 4 vector 88 msi: routing MSI IRQ 47 to local APIC 4 vector 89 msi: routing MSI IRQ 48 to local APIC 4 vector 90 msi: routing MSI IRQ 49 to local APIC 4 vector 91 msi: routing MSI IRQ 50 to local APIC 4 vector 92 msi: routing MSI IRQ 51 to local APIC 4 vector 93 msi: routing MSI IRQ 52 to local APIC 4 vector 94 msi: routing MSI IRQ 53 to local APIC 4 vector 95 msi: routing MSI IRQ 54 to local APIC 4 vector 96 msi: routing MSI IRQ 55 to local APIC 4 vector 97 msi: routing MSI IRQ 56 to local APIC 4 vector 98 msi: routing MSI IRQ 57 to local APIC 4 vector 99 msi: routing MSI IRQ 58 to local APIC 4 vector 100 msi: routing MSI IRQ 59 to local APIC 4 vector 101 msi: routing MSI IRQ 60 to local APIC 4 vector 102 msi: routing MSI IRQ 61 to local APIC 4 vector 103 msi: routing MSI IRQ 62 to local APIC 4 vector 104 msi: routing MSI IRQ 63 to local APIC 4 vector 105 msi: routing MSI IRQ 64 to local APIC 4 vector 106 msi: routing MSI IRQ 65 to local APIC 4 vector 107 msi: routing MSI IRQ 66 to local APIC 4 vector 108 msi: routing MSI IRQ 67 to local APIC 4 vector 109 msi: routing MSI IRQ 68 to local APIC 4 vector 110 msi: routing MSI IRQ 69 to local APIC 4 vector 111 timecard0: using IRQs 38-69 for MSI TimeCard msi 32 rman iomem start f4000000, end f5ffffff, size 2000000 timecard_corelist_read did not find list, using fb list 1 nports 1 Start with uart 0 0 irq 41, irid 4 Fatal trap 30: reserved (unknown) fault while in kernel mode cpuid = 2; apic id = 04 port irq rid 4 instruction pointer = 0x20:0xffffffff804dda2d stack pointer = 0x28:0xfffffe001b1d8de0 TimeCard device loaded. pci2: driver added pci3: driver added pci4: driver added frame pointer = 0x28:0xfffffe001b1d8e10 code segment = base 0x0, limit 0xfffff, type 0x1b pci5: driver added = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, IOPL = 0 current process = 11 (idle: cpu2) trap number = 30 panic: reserved (unknown) fault cpuid = 2 time = 1695375919 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe001b1d8c00 vpanic() at vpanic+0x151/frame 0xfffffe001b1d8c50 panic() at panic+0x43/frame 0xfffffe001b1d8cb0 trap_fatal() at trap_fatal+0x387/frame 0xfffffe001b1d8d10 calltrap() at calltrap+0x8/frame 0xfffffe001b1d8d10 --- trap 0x1e, rip = 0xffffffff804dda2d, rsp = 0xfffffe001b1d8de0, rbp = 0xfffffe001b1d8e10 --- acpi_cpu_idle() at acpi_cpu_idle+0x28d/frame 0xfffffe001b1d8e10 cpu_idle_acpi() at cpu_idle_acpi+0x4d/frame 0xfffffe001b1d8e30 cpu_idle() at cpu_idle+0xac/frame 0xfffffe001b1d8e50 sched_idletd() at sched_idletd+0x4b1/frame 0xfffffe001b1d8ef0 fork_exit() at fork_exit+0x80/frame 0xfffffe001b1d8f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe001b1d8f30 --- trap 0xe0c2545f, rip = 0x8f4a0c76246ffed9, rsp = 0x670c1cf091ea5e5b, rbp = 0x57639a151e2b2985 --- KDB: enter: panic [ thread pid 11 tid 100005 ] Stopped at kdb_enter+0x37: movq $0,0x129840e(%rip) db> x/wx timecard_intr_count timecard_intr_count: 0 db> show apic Interrupts bound to lapic 0 vec 0x30 -> IRQ 9 vec 0x31 -> IRQ 28 vec 0x32 -> IRQ 31 vec 0x33 -> IRQ 24 vec 0x34 -> IRQ 29 vec 0x35 -> IRQ 30 vec 0x36 -> IRQ 8 vec 0x37 -> IRQ 16 vec 0x38 -> IRQ 36 vec 0x39 -> IRQ 1 vec 0xef -> lapic timer Interrupts bound to lapic 2 vec 0x30 -> IRQ 25 vec 0x31 -> IRQ 33 vec 0x32 -> IRQ 23 vec 0x33 -> IRQ 18 vec 0xef -> lapic timer Interrupts bound to lapic 4 vec 0x30 -> IRQ 26 vec 0x31 -> IRQ 0 vec 0x32 -> IRQ 34 vec 0x33 -> IRQ 37 vec 0x50 -> IRQ 38 vec 0x51 -> IRQ 39 vec 0x52 -> IRQ 40 vec 0x53 -> IRQ 41 vec 0x54 -> IRQ 42 vec 0x55 -> IRQ 43 vec 0x56 -> IRQ 44 vec 0x57 -> IRQ 45 vec 0x58 -> IRQ 46 vec 0x59 -> IRQ 47 vec 0x5a -> IRQ 48 vec 0x5b -> IRQ 49 vec 0x5c -> IRQ 50 vec 0x5d -> IRQ 51 vec 0x5e -> IRQ 52 vec 0x5f -> IRQ 53 vec 0x60 -> IRQ 54 vec 0x61 -> IRQ 55 vec 0x62 -> IRQ 56 vec 0x63 -> IRQ 57 vec 0x64 -> IRQ 58 vec 0x65 -> IRQ 59 vec 0x66 -> IRQ 60 vec 0x67 -> IRQ 61 vec 0x68 -> IRQ 62 vec 0x69 -> IRQ 63 vec 0x6a -> IRQ 64 vec 0x6b -> IRQ 65 vec 0x6c -> IRQ 66 vec 0x6d -> IRQ 67 vec 0x6e -> IRQ 68 vec 0x6f -> IRQ 69 vec 0xef -> lapic timer Interrupts bound to lapic 6 vec 0x30 -> IRQ 27 vec 0x31 -> IRQ 32 vec 0x32 -> IRQ 35 vec 0x33 -> IRQ 4 vec 0xef -> lapic timer db> show lapic lapic ID = 4 version = 1.5 max LVT = 6 SVR = ff (enabled) TPR = 00 In-service Interrupts: isr2: 43 TMR Interrupts: IRR Interrupts: irr1: 32 33 irr2: 43 irr7: ef db> </snip> Regards John