9300-16e does not see 6G shelf

[Alan Somers]

On Fri, May 1, 2015 at 4:52 PM, John <jwd at freebsd.org> wrote:
> Hi Folks,
>
>    I'm running 10-STABLE with some LSI (Avago) 9300-16e HBA
> cards with the mpr driver. These cards work great with 12G
> shelves (system & shelves from SuperMicro).
>
>    However, if I hook a set of 6G shelves from a system
> using LSI 9207-8e HBA cards (a nicely working config), the mpr
> driver complains that no active slots are found and the shelf is
> not registered - leading to no drives being discovered.
>
>    I am not mixing speeds on the bus. I'm using only 6G shelves
> in this configuration.
>
>    The shelf information from a 9207 based system:
>
> ses0 at mps1 bus 0 scbus7 target 169 lun 0
> ses0: <HP D2700 SAS AJ941A 0131> Fixed Enclosure Services SCSI-5 device
> ses0: 600.000MB/s transfers
> ses0: Command Queueing enabled
> ses0: SCSI-3 SES Device
>
>    From the system with the 9300-16e cards (verbose boot):
>
> mpr0: <LSI SAS3008> port 0xf000-0xf0ff mem 0xfbd40000-0xfbd4ffff,0xfbd00000-0xfbd3ffff irq 56 at device 0.0 on pci132
> mpr0: IOCFacts  :
>         MsgVersion: 0x205
>         HeaderVersion: 0x2300
>         IOCNumber: 0
>         IOCExceptions: 0x0
>         MaxChainDepth: 128
>         NumberOfPorts: 1
>         RequestCredit: 10112
>         ProductID: 0x2221
>         IOCRequestFrameSize: 32
>         MaxInitiators: 32
>         MaxTargets: 1024
>         MaxSasExpanders: 64
>         MaxEnclosures: 65
>         HighPriorityCredit: 120
>         MaxReplyDescriptorPostQueueDepth: 65504
>         ReplyFrameSize: 32
>         MaxVolumes: 0
>         MaxDevHandle: 1128
>         MaxPersistentEntries: 128
> mpr0: Firmware: 08.00.00.00, Driver: 05.255.05.00-fbsd
> mpr0: IOCCapabilities: 7a85c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,MSIXIndex,HostDisc>
> mpr0: attempting to allocate 1 MSI-X vectors (96 supported)
> msi: routing MSI-X IRQ 302 to local APIC 0 vector 100
> mpr0: using IRQ 302 for MSI-X
>
>
>    and then the following:
>
>
> mpr_mapping_enclosure_dev_status_change_event: enclosure with handle = 0x2 reported 0 slots
> _mapping_get_dev_info: failed to add the device with handle 0x000a because the enclosure is not in the mapping table
> _mapping_get_dev_info: failed to add the device with handle 0x000b because the enclosure is not in the mapping table
> _mapping_get_dev_info: failed to add the device with handle 0x000c because the enclosure is not in the mapping table
> _mapping_get_dev_info: failed to add the device with handle 0x000d because the enclosure is not in the mapping table
> _mapping_get_dev_info: failed to add the device with handle 0x000e because the enclosure is not in the mapping table
>
>    Followed by many more of the above messages for each of the disks.
>
>    I've started looking at the Mpi2EventDataSasEnclDevStatusChange_t
> structure - and I'm not convinced it's filled out fully.
>
>    I can drop the 9207 cards in the new system and things
> work fine - but I would like to use the newer cards if possible.
>
>    Is there something I've missed? A configuration knob?
>
>    Thoughts?
>
> Thanks,
> John

This smells like another Slot Mapping problem.  When you connect the
6G shelf, do you see any devices at all?  I would expect that you see
at least one, though it might be ses(4) instead of da(4).  If this is
the Slot Mapping problem, then it doesn't really have anything to do
with 12G vs 6G.  Rather, it's an issue with specific how SAS expanders
report their phy info, and how the HBA firmware interprets it.  The
HBA firmware can use two different modes: persistent mapping or
enclosure/slot mapping.  By default, LSI's 6G HBAs used persistent
mapping, but their 12G HBAs use device/slot mapping by default.

This post explains how to change the HBA's mode.

https://lists.freebsd.org/pipermail/freebsd-scsi/2014-December/006545.html

-Alan