ses/pass devices (enclosure/processor devices) not all showing up?

[John De Boskey]

----- Alan Somers's Original Message -----
> On Tue, Sep 8, 2015 at 9:35 PM, John De Boskey <jwd at freebsd.org> wrote:
> > Hi Folks -
> >
> >    I have a shelf with 84 sata drives. All drives show up
> > correctly and are accessible. The shelf appears to have
> > multiple processor devices and one enclosure device internally.
> > For instance:
> >
> > # camcontrol devlist | grep XYRATEX
> > <XYRATEX DEFAULT-SD-R24 3034>      at scbus7 target 159 lun 0 (pass18)
> > <XYRATEX DEFAULT-SD-R36 3034>      at scbus7 target 188 lun 0 (pass47)
> > <XYRATEX DEFAULT-SD-R36 3034>      at scbus7 target 217 lun 0 (pass76)
> > <XYRATEX DEFAULT-SD-R24 3034>      at scbus7 target 232 lun 0 (pass91)
> >
> > # camcontrol devlist | grep ses
> > <DELL SC280-01-E6EBD 3034>         at scbus7 target 144 lun 0 (ses0,pass3)
> >
> > # camcontrol smprg pass18 | grep 'Number of Phys:'
> > Number of Phys: 25
> > # camcontrol smprg pass47 | grep 'Number of Phys:'
> > Number of Phys: 37
> > # camcontrol smprg pass76 | grep 'Number of Phys:'
> > Number of Phys: 37
> > # camcontrol smprg pass91 | grep 'Number of Phys:'
> > Number of Phys: 25
> > # camcontrol smprg ses0 | grep 'Number of Phys:'
> > Number of Phys: 37
> >
> > # camcontrol smpphylist pass18
> > 25 PHYs:
> > PHY  Attached SAS Address
> >   0  0x5000c500585f4b52   <SEAGATE ST4000NM0023 GE09>       (pass4,da0)

... 

> > I have linked dmesg, camcontrol devlist, and sas2ircu output below:
> >
> > http://people.freebsd.org/~jwd/sespass/dmesg.txt   mps messaging enabled.
> >
> > http://people.freebsd.org/~jwd/sespass/devlist.txt
> >
> > http://people.freebsd.org/~jwd/sespass/sas2ircu.txt
> >
> > Thanks,
> > John

Hi Alan,

> I'm not sure exactly what you're asking.  Are you just wondering why

   In a simple form, I'm trying to figure out why 'camcontrol smpphyslist ses0'
produces no output, while 'camcontrol smpphyslist pass18' does.

> your pass devices don't also have ses nodes?  I think I know why.
> Your JBOD problem has five SAS expander chips, though ses0 might
> actually be some other kind of SAS target chip.  pass18, pass47,
> pass76, and pass91 are configured to report ses0's SAS Address as the
> address of their SEP.  The LSI HBA's interpretation is that there is

   Could you expand on the concept of the SAS Address being reported?
When I look at the address of the devices in the sas2ircu output they
are unique.  For example:

Device is a Processor device
  Enclosure #                             : 2
  Slot #                                  : 1
  SAS Address                             : 50050cc-1-0d2f-fabe
  State                                   : Standby (SBY)
  Manufacturer                            : XYRATEX 
  Model Number                            : DEFAULT-SD-R24  
  Firmware Revision                       : 3034
  Serial No                               : PMCSIERA

Device is a Processor device
  Enclosure #                             : 2
  Slot #                                  : 2
  SAS Address                             : 50050cc-1-0d2f-fafe
  State                                   : Standby (SBY)
  Manufacturer                            : XYRATEX 
  Model Number                            : DEFAULT-SD-R36  
  Firmware Revision                       : 3034
  Serial No                               : PMCSIERA

   ie: they show as unique. However, they don't appear to be unique
as reported by camcontrol smprg. I haven't been able to determine
where in the code this linkage is being done yet.


> only one SES processor.  So FreeBSD reports one ses device, and the
> other expanders just show up as pass devices.  Unless the manufacturer
> royally screwed up (I doubt it), ses0 will report info for all 84
> disks in its SES status pages.

camcontrol smpphylist ses0 - returns no information.
camcontrol smpphylist pass18 - returns a drive list
camcontrol smpphylist pass47 - returns a drive list
camcontrol smpphylist pass76 - returns a drive list
camcontrol smpphylist pass91 - returns a drive list

> Bill is right, sg3_utils is your friend.  However, I doubt you'll see
> any secondary subenclosures.  That feature isn't much used, and it's
> not necessary in order for ses0 to report all 84 drives.  Running
> these two commands will probably tell you most of what you need to
> know:
> 
> sg_ses -p 1 ses0
> sg_ses -p 2 ses0

   I have run the above commands and linked the output below.

http://people.freebsd.org/~jwd/sespass/sgsesp1.txt  

http://people.freebsd.org/~jwd/sespass/sgsesp2.txt   descriptor for each disk

   Thoughts?

Thanks,
John

> -Alan