[Bug 263223] MPR driver fails to attach on Dell Precision 5810 with "mpr0: IOC Facts allocation failed with error 12"

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=263223

            Bug ID: 263223
           Summary: MPR driver fails to attach on Dell Precision 5810 with
                    "mpr0: IOC Facts allocation failed with error 12"
           Product: Base System
           Version: CURRENT
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: rwarning@atto.com

Created attachment 233149
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=233149&action=edit
Tarball described in the comments

I'm a developer working for ATTO technology, currently looking into the
feasibility of adding support for our H12XXGT adapters to the FreeBSD kernel.
These adapters use an LSI (Broadcom now) MPT-3 fusion chip-set that is
supported by the mpr driver.

When adding our PCI IDs to the mpr driver I found that our adapter was failing
to attach due to bus_dmamem_alloc failing in mpr_alloc_nvme_prp_pages of mpr.c
of the mpr driver. Further investigation showed that DMA memory allocations are
restricted to the first 4Gb of memory on the system. On my particular system
mpr_alloc_nvme_prp_pages was attempting to allocate ~512Mb of contiguous
physical memory and could find nothing within the first 4Gb to satisfy this.

Our board does not have this restriction. To the best of my knowledge, neither
does any other MPT-3 fusion chipset. The restriction is that DMA memory must be
within a 4Gb window, must not cross a 4Gb boundary, but can be anywhere within
a 64bit address space.

Attached is a patch that demonstrates a fix for this issue. The patch removes
the lower address limit of BUS_SPACE_MAXADDR_32BIT and replaces it with a 4Gb
boundary limit. I ran some preliminary tests with the patch on one of our
H1280GTs and a Broadcom SAS3816. IO and data integrity appear to be fine. I
have not yet tested NVMe functionality, nor have I tested that sense data is
being reported correctly.

The patch is still incomplete as a fix. There's some chip diagnostics that I'm
unfamiliar with in mpr_user.c that needs to be updated. I also noticed from the
busdma(9) man page that the memory boundary limits must not exceed the maximum
segment size. That is not done in the patch. I'm also sure I overlooked
something in the style guidelines.

For simplicity's sake, the patch is also missing PCI IDs for the H1280GT.

Attached is a tarball with:

1. The patch itself (patch.diff)
2. Output of uname -a (uname.txt)
3. /var/log/messages output of the mpr driver failing to attach to our H1280GT
(messages.bad)
4. /var/log/messages output of the mpr driver attaching to our H1280GT with the
patch applied (messages.good)

Both message files have a Broadcom SAS3816 in the system for reference. Both
have debug output for the mpr driver cranked up to the maximum setting.

-- 
You are receiving this mail because:
You are the assignee for the bug.