mps questions

Thu Jan 20 17:00:48 UTC 2011

On Thu, Jan 20, 2011 at 17:47:21 +0100, Borja Marcos wrote:
> 
> Hello,
> 
> I''m trying some Dell PERC H200 cards on FreeBSD 8.2 with v28 ZFS. (Not a production environment, just running tests).
> 
> I have two variants of them. One is an internal one, identified as this one.
> 
> mps0: <LSI SAS2008> port 0xfc00-0xfcff mem 0xdf2b0000-0xdf2bffff,0xdf2c0000-0xdf2fffff irq 32 at device 0.0 on pci2
> mps0: Firmware: 02.15.63.00
> mps0: IOCCapabilities: 185c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,IR>
> mps0: [ITHREAD]
> 
> 
> And another one intended to conect to external SAS devices.
> 
> mps1: <LSI SAS2008> port 0xec00-0xecff mem 0xdf4b0000-0xdf4bffff,0xdf4c0000-0xdf4fffff irq 41 at device 0.0 on pci5
> mps1: Firmware: 02.15.63.00
> mps1: IOCCapabilities: 185c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,IR>
> mps1: [ITHREAD]
> 
> Both have two SAS ports. And I am trying them on Dell PowerEdge R510 servers. I have 8 internal hard disks attached, and a Dell PowerVault MD1220 connected to one of the external SAS connectors. The MD1220 has 8 hard disks as well.
> 
> I haven't configured any logical volumes on the cards, I just want unobtrusive SAS cards in order to create ZFS pools on raw disks, and the setup is working great, except for one problem:
> 
> The *internal* card is showing just six hard disks under FreeBSD, not eight.
> 
> da0 at mps0 bus 0 scbus0 target 1 lun 0
> da1 at mps0 bus 0 scbus0 target 2 lun 0
> da2 at mps0 bus 0 scbus0 target 3 lun 0
> da3 at mps0 bus 0 scbus0 target 4 lun 0
> da4 at mps0 bus 0 scbus0 target 5 lun 0
> da5 at mps0 bus 0 scbus0 target 6 lun 0
> 
> And this happens no matter how I connect the internal SAS connectors. There are two connectors from the disk backplanes to the card, SAS0 and SAS1 (I assume they are two buses) and even if I reverse them I still see eight disks. 
> 

I assume you mean you still see six disks...

> However, when booting the system I see eight disks. The controller  BIOS lists the backplane device and the eight disks, the FreeBSD boot loader says it sees eight disks, but FreeBSD sees just six.
> 
> # camcontrol rescan all
> Re-scan of bus 0 was successful
> Re-scan of bus 1 was successful
> # camcontrol devlist
> <SEAGATE ST3300657SS-H EH02>       at scbus0 target 1 lun 0 (da0,pass0)
> <SEAGATE ST3300657SS-H EH02>       at scbus0 target 2 lun 0 (da1,pass1)
> <SEAGATE ST3300657SS-H EH02>       at scbus0 target 3 lun 0 (da2,pass2)
> <SEAGATE ST3300657SS-H EH02>       at scbus0 target 4 lun 0 (da3,pass3)
> <SEAGATE ST3300657SS-H EH02>       at scbus0 target 5 lun 0 (da4,pass4)
> <SEAGATE ST3300657SS-H EH02>       at scbus0 target 6 lun 0 (da5,pass5)
> <Alcor Flash Disk 8.07>            at scbus1 target 0 lun 0 (da6,pass6)
> 
> The card with the outside facing connectors shows the eight hard disks atttached to the MD1120 and the disks. A camcontrol devlist on the machine with both mps cards installed I see this:
> 
> # camcontrol devlist
> <SEAGATE ST3300657SS-H EH02>       at scbus0 target 1 lun 0 (da0,pass0)
> <SEAGATE ST3300657SS-H EH02>       at scbus0 target 2 lun 0 (da1,pass1)
> <SEAGATE ST3300657SS-H EH02>       at scbus0 target 3 lun 0 (da2,pass2)
> <SEAGATE ST3300657SS-H EH02>       at scbus0 target 4 lun 0 (da3,pass3)
> <SEAGATE ST3300657SS-H EH02>       at scbus0 target 5 lun 0 (da4,pass4)
> <SEAGATE ST3300657SS-H EH02>       at scbus0 target 6 lun 0 (da5,pass5)
> <SEAGATE ST9146852SS HT64>         at scbus1 target 1 lun 0 (da6,pass6)
> <SEAGATE ST9146852SS HT64>         at scbus1 target 2 lun 0 (da7,pass7)
> <SEAGATE ST9146852SS HT64>         at scbus1 target 3 lun 0 (da8,pass8)
> <SEAGATE ST9146852SS HT64>         at scbus1 target 4 lun 0 (da9,pass9)
> <SEAGATE ST9146852SS HT64>         at scbus1 target 5 lun 0 (da10,pass10)
> <SEAGATE ST9146852SS HT64>         at scbus1 target 6 lun 0 (da11,pass11)
> <SEAGATE ST9146852SS HT64>         at scbus1 target 7 lun 0 (da12,pass12)
> <SEAGATE ST9146852SS HT64>         at scbus1 target 13 lun 0 (da13,pass13)
> <DELL MD1220 1.01>                 at scbus1 target 36 lun 0 (ses0,pass14)
> <UT165 USB Flash Disk 0.00>        at scbus2 target 0 lun 0 (pass15,da14)
> 
> 
> Any ideas? I was wondering if Dell had done something nasty to the internal card firmware (mps0 in my example), but seems that the eight internal disks are correctly presented to the BIOS.
> 

If you see all the disks in the LSI BIOS, it most likely means there is a
driver bug that is keeping FreeBSD from seeing all the disks.

Try booting with -v (boot -v at the boot loader prompt) and send the full
dmesg output.  That will cause the driver to print some additional
information when it probes.

After that, try booting with hw.mps.N.debug_level=1 set in loader.conf,
where N is either 0 or 1, depending on which mps instance is having the
problem.

You can turn it off via sysctl after you boot.

Ken
-- 
Kenneth Merry
ken at FreeBSD.ORG