LSI2008 controller clobbers first disk with new LSI mps driver

Fri Feb 17 06:29:31 UTC 2012

Jason,

I have gone through your data provided in this thread. It is well understood because of your descriptive data.

So What I understood here is:

1. You tested with HBA Fw "07.00.00.00" and "10.00.02.00"

2. you have run your test on two different LSI BIOS versions.
Grabbed from below line.
MPT2BIOS-7.11.00.00 (2010.07.29) / PRODUCT REVISION 7.00.00.00
MPT2BIOS-7.19.00.00 (2011.05.16) / PRODUCT REVISION 10.00.02.00

Now I am able to see below three difference in your setup.

See FW version and check starting target id, all three has different way of assigning TargetIDs.
For first two case target id start with "8" but SES device assignment is different.
Last case target id start with "1"

mps0: Firmware: 07.00.00.00, Driver: 11.255.03.00-fbsd (OR) 13.00.00.00-fbsd
> > <SEAGATE ST91000640SS 0001> at scbus0 target 8 lun 0 (pass0,da0)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 9 lun 0 (pass1,da1)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 10 lun 0 (pass2,da2)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 11 lun 0 (pass3,da3)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 12 lun 0 (pass4,da4)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 13 lun 0 (pass5,da5)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 14 lun 0 (pass6,da6)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 15 lun 0 (pass7,da7)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 16 lun 0 (pass8,da8)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 17 lun 0 (pass9,da9)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 18 lun 0 (pass10,da10)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 19 lun 0 (pass11,da11)
> > <LSI CORP SAS2X28 0717> at scbus0 target 20 lun 0 (ses0,pass12)

mps0: Firmware: 10.00.02.00, Driver: 13.00.00.00-fbsd
mps0: Firmware: 10.00.02.00, Driver: 11.00.00.00 (OR) 8.2-STABLE Inbox
> > <LSI CORP SAS2X28 0717> at scbus0 target 8 lun 0 (ses0,pass0)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 9 lun 0 (da0,pass1)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 10 lun 0 (da1,pass2)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 11 lun 0 (da2,pass3)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 12 lun 0 (da3,pass4)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 13 lun 0 (da4,pass5)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 14 lun 0 (da5,pass6)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 15 lun 0 (da6,pass7)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 16 lun 0 (da7,pass8)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 17 lun 0 (da8,pass9)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 18 lun 0 (da9,pass10)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 19 lun 0 (da10,pass11)

On the FBSD developed driver active in 8-STABLE prior to the LSI
Release (Firmware: 10.00.02.00)

> > <SEAGATE ST91000640SS 0001> at scbus0 target 1 lun 0 (pass0,da0)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 2 lun 0 (pass1,da1)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 3 lun 0 (pass2,da2)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 4 lun 0 (pass3,da3)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 5 lun 0 (pass4,da4)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 6 lun 0 (pass5,da5)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 7 lun 0 (pass6,da6)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 8 lun 0 (pass7,da7)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 9 lun 0 (pass8,da8)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 10 lun 0 (pass9,da9)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 11 lun 0 (pass10,da10)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 12 lun 0 (pass11,da11)
> > <LSI CORP SAS2X28 0717> at scbus0 target 13 lun 0 (ses0,pass12)

In summary,  (please confirm)
1. you have not seen any issue if you use "07.00.00.00" FW version.
2. _but_ when you use "10.00.02.00" FW, with "13.00.00.00-fbsd" driver version you are seeing 
SES is detected before Drives as pass0.
3. When you use "10.00.02.00" FW with 8-STABLE inbox FBSD driver, you are finding SES device detected after Drives.

All driver is doing here is asking CAM layer to scan Bus when there is any device added on that bus.
So depending upon actual target Id  assigned by FW, it will be detected to camcontrol.

So bottom line is FW plays major role in sequencing Drives behind LSI controller. 

~ Kashyap

> -----Original Message-----
> From: owner-freebsd-scsi at freebsd.org [mailto:owner-freebsd-
> scsi at freebsd.org] On Behalf Of Jason Wolfe
> Sent: Friday, February 17, 2012 8:09 AM
> To: freebsd-scsi at freebsd.org
> Subject: Re: LSI2008 controller clobbers first disk with new LSI mps
> driver
> 
> Noticed there were 2 patches made recently to mps, tested with those
> also
> with the same boot drive masking results:
> 
> 8.3-PRERELEASE / mps0: Firmware: 10.00.02.00, Driver: 13.00.00.00-fbsd
> 
> Jason
> 
> On Thu, Feb 16, 2012 at 1:46 PM, Jason Wolfe <nitroboost at gmail.com>
> wrote:
> 
> > I'm having an issue with the recent and older LSI developed mps
> drivers
> > (r230922) on the LSI2008 SAS2 controller running IT firmware
> (Supermicro
> > X8DTT-H) where the controller (ses0) is seen up as the first device
> (pass0)
> > and clobbers the boot drive, leaving only 11 of the 12 drives picked
> up by
> > the OS. When using the FBSD developed driver that had been in 8.2-
> STABLE
> > for the past year up until the recent LSI release earlier this month,
> the
> > controller is picked up last and all 12 drives show properly.
> >
> > In all cases BIOS and the boot loader see all drives properly, and
> again
> > the FBSD dev driver sees all 12 in the OS. This causes the system to
> boot
> > with the LSI drivers, but when it goes to mount to the root it hangs
> at the
> > prompt. I've reverted all my systems back to the older FBSD dev driver
> for
> > now so I dont have to boot from the 2nd drive, but I'm game to revert
> and
> > do any testing.
> >
> > LSI Corporation MPT SAS2 BIOS
> > MPT2BIOS-7.19.00.00 (2011.05.16)
> > Copyright 2000-2011 LSI Corporation.
> >
> > PCI ENCL LUN VENDOR PRODUCT PRODUCT SIZE
> > SLOT SLOT NUM NAME IDENTIFIER REVISION NVDATA
> > ---- ---- --- -------- ---------------- ----------- ---------
> > 5 LSI SAS2008-IT 10.00.02.00 0A:02:00:04
> > 5 0 0 SEAGATE ST91000640SS 0001 953 GB
> > 5 1 0 SEAGATE ST91000640SS 0001 953 GB
> > 5 2 0 SEAGATE ST91000640SS 0001 953 GB
> > 5 3 0 SEAGATE ST91000640SS 0001 953 GB
> > 5 4 0 SEAGATE ST91000640SS 0001 953 GB
> > 5 5 0 SEAGATE ST91000640SS 0001 953 GB
> > 5 6 0 SEAGATE ST91000640SS 0001 953 GB
> > 5 7 0 SEAGATE ST91000640SS 0001 953 GB
> > 5 8 0 SEAGATE ST91000640SS 0001 953 GB
> > 5 9 0 SEAGATE ST91000640SS 0001 953 GB
> > 5 10 0 SEAGATE ST91000640SS 0001 953 GB
> > 5 11 0 SEAGATE ST91000640SS 0001 953 GB
> >
> >
> > BIOS drive C: is disk0
> > BIOS drive D: is disk1
> > BIOS drive E: is disk2
> > BIOS drive F: is disk3
> > BIOS drive G: is disk4
> > BIOS drive H: is disk5
> > BIOS drive I: is disk6
> > BIOS drive J: is disk7
> > BIOS drive K: is disk8
> > BIOS drive L: is disk9
> > BIOS drive M: is disk10
> > BIOS drive N: is disk11
> >
> >
> > Running the LSI developed binary (11.00.00.00) on/for 7.2-RELEASE or
> LSI
> > developed driver (released to the community earlier this month) in
> > 8.2-STABLE:
> > <LSI CORP SAS2X28 0717> at scbus0 target 8 lun 0 (ses0,pass0)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 9 lun 0 (da0,pass1)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 10 lun 0 (da1,pass2)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 11 lun 0 (da2,pass3)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 12 lun 0 (da3,pass4)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 13 lun 0 (da4,pass5)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 14 lun 0 (da5,pass6)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 15 lun 0 (da6,pass7)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 16 lun 0 (da7,pass8)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 17 lun 0 (da8,pass9)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 18 lun 0 (da9,pass10)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 19 lun 0 (da10,pass11)
> >
> >
> > On the FBSD developed driver active in 8-STABLE prior to the LSI
> release:
> >
> > <SEAGATE ST91000640SS 0001> at scbus0 target 1 lun 0 (pass0,da0)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 2 lun 0 (pass1,da1)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 3 lun 0 (pass2,da2)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 4 lun 0 (pass3,da3)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 5 lun 0 (pass4,da4)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 6 lun 0 (pass5,da5)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 7 lun 0 (pass6,da6)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 8 lun 0 (pass7,da7)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 9 lun 0 (pass8,da8)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 10 lun 0 (pass9,da9)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 11 lun 0 (pass10,da10)
> > <SEAGATE ST91000640SS 0001> at scbus0 target 12 lun 0 (pass11,da11)
> > <LSI CORP SAS2X28 0717> at scbus0 target 13 lun 0 (ses0,pass12)
> >
> > Our one lead is that in older machines of the same model but older
> > firmware, this behavior is _NOT_ seen.
> >
> > Supermicro X8DTT-H BIOS Date: 02/11/11 14:48:51 Ver 2.0c
> >
> > LSI Corporation MPT SAS2 BIOS
> > MPT2BIOS-7.11.00.00 (2010.07.29)
> > Copyright 2000-2010 LSI Corporation.
> >
> > PCI ENCL LUN VENDOR PRODUCT PRODUCT SIZE
> > SLOT SLOT NUM NAME IDENTIFIER REVISION NVDATA
> > ---- ---- --- -------- ---------------- ----------- ---------
> > 5 LSI Corp SAS2008-IT 7.00.00.00 07:00:00:05
> > 5 0 0 SEAGATE ST91000640SS 0001 931 GB
> > 5 1 0 SEAGATE ST91000640SS 0001 931 GB
> > 5 2 0 SEAGATE ST91000640SS 0001 931 GB
> > 5 3 0 SEAGATE ST91000640SS 0001 931 GB
> > 5 4 0 SEAGATE ST91000640SS 0001 931 GB
> > 5 5 0 SEAGATE ST91000640SS 0001 931 GB
> > 5 6 0 SEAGATE ST91000640SS 0001 931 GB
> > 5 7 0 SEAGATE ST91000640SS 0001 931 GB
> > 5 8 0 SEAGATE ST91000640SS 0001 931 GB
> > 5 9 0 SEAGATE ST91000640SS 0001 931 GB
> > 5 10 0 SEAGATE ST91000640SS 0001 931 GB
> > 5 11 0 SEAGATE ST91000640SS 0001 931 GB
> >
> > So works fine:
> >
> > MPT2BIOS-7.11.00.00 (2010.07.29) / PRODUCT REVISION 7.00.00.00
> >
> > Clobbers boot drive:
> >
> > MPT2BIOS-7.19.00.00 (2011.05.16) / PRODUCT REVISION 10.00.02.00
> >
> > On both system types the hardware/capabilities are identical, only
> > variation is the LSI drivers print the Driver: X after the Firmware,
> where
> > the FBSD drive did not:
> >
> > mps0: <LSI SAS2008> port 0xe000-0xe0ff mem
> > 0xfbd3c000-0xfbd3ffff,0xfbd40000-0xfbd7ffff irq 26 at device 0.0 on
> pci4
> > mps0: Firmware: 07.00.00.00, Driver: 11.255.03.00-fbsd
> > mps0: IOCCapabilities:
> >
> 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDis
> c>
> > mps0: [ITHREAD]
> >
> > ses0 at mps0 bus 0 scbus0 target 20 lun 0
> > ses0: <LSI CORP SAS2X28 0717> Fixed Enclosure Services SCSI-5 device
> > ses0: 600.000MB/s transfers
> > ses0: Command Queueing enabled
> > ses0: SCSI-3 SES Device
> >
> > mps0 at pci0:4:0:0: class=0x010700 card=0x040015d9 chip=0x00721000
> rev=0x02
> > hdr=0x00
> > vendor = 'LSI Logic (Was: Symbios Logic, NCR)'
> > class = mass storage
> > subclass = SAS
> >
> >
> > Thanks in advance for any advice, I'm guessing a bug is being tickled
> in
> > the driver by the new mpt firmware, or vice versa? Might it be
> possible to
> > use hints to force the controller to be picked up last?
> >
> > Jason
> >
> _______________________________________________
> freebsd-scsi at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-scsi
> To unsubscribe, send any mail to "freebsd-scsi-unsubscribe at freebsd.org"