OT-ish SATA port replicators vs. SAS "expanders"

[Mike Tancsa]

On 11/17/2014 5:50 PM, John-Mark Gurney wrote:
> Brandon Vincent wrote this message on Sun, Nov 16, 2014 at 17:26 -0700:
>> On Sun, Nov 16, 2014 at 4:51 PM, Mike Tancsa <mike at sentex.net> wrote:
>>> I don think a port multiplier is the same as a SAS expander and a SAS/SATA
>>> controller does not necessarily support port multipliers.
>>
>> You're absolutely correct. The SATA specifications if I recall
>> correctly, outline that providing support for port multipliers is
>> optional. Not to mention, the vast majority of chipsets that do have
>> support for port multipliers only provide the slower command-based
>> switching method [1], The LSI SAS 2008 chipset is not what you
>> typically find on the cheap consumer "fake RAID" cards which most
>> people have [2].
>>
>> Most people who purchase this card are purchasing it for SAS, true
>> RAID, performance, and reliability. Thus, LSI is only going to provide
>> support for the far superior SAS expander technology.
>
> If you go w/ SATA PMs, make sure you get a controller that supports
> FIS-base switching instead of command based switching...  You'll get
> much better performance w/ NCQ support...   As for if the LSI SAS 2008
> will support FBS, probably not, as it'll require the driver and/or
> firmware to be aware of how to handle things...

Performance was pretty good, but reliability was really poor for us. 
Not sure if it was the cages, cables, or a firmware issue in the card 
(SiI3132) or cages or a driver issue.  But we would regularly see errors 
such as

  kernel: siisch0: Timeout on slot 18
  kernel: siisch0: siis_timeout is 00040000 ss 001ff800 rs 001ff800 es 
00000000 sts 80112000 serr 00000000
  kernel: siisch0:  ... waiting for slots 001bf800
  kernel: siisch0: Timeout on slot 19
  kernel: siisch2: Error while READ LOG EXT
  kernel: (ada10:siisch2:0:2:0): WRITE_FPDMA_QUEUED. ACB: 61 ab 95 98 71 
40 b6 00 00 00 00 00
  kernel: (ada10:siisch2:0:2:0): CAM status: ATA Status Error
  kernel: (ada10:siisch2:0:2:0): ATA status: 00 ()
  kernel: (ada10:siisch2:0:2:0): RES: 00 00 00 00 00 00 00 00 00 00 00
  kernel: (ada10:siisch2:0:2:0): Retrying command
  kernel: siisch2: Error while READ LOG EXT


But after having swapped things far too many times, we switched to a 
pair of LSI 9240-8i, and using the same disks (wiped and started fresh), 
we have yet to get any of the old errors

prior to re-using the disks (there were no SMART errors logged), we did 
a long test as well as wrote to every sector (no errors).   Two weeks 
later, still no errors. Even better performance too.

   pool: zbackup1
  state: ONLINE
   scan: none requested
config:

         NAME        STATE     READ WRITE CKSUM
         zbackup1    ONLINE       0     0     0
           raidz2-0  ONLINE       0     0     0
             da0p1   ONLINE       0     0     0
             da1p1   ONLINE       0     0     0
             da2p1   ONLINE       0     0     0
             da3p1   ONLINE       0     0     0
             da4p1   ONLINE       0     0     0
             da5p1   ONLINE       0     0     0
           raidz2-2  ONLINE       0     0     0
             da6     ONLINE       0     0     0
             da7     ONLINE       0     0     0
             da8     ONLINE       0     0     0
             da9     ONLINE       0     0     0
             da10    ONLINE       0     0     0
             da11    ONLINE       0     0     0
         logs
           ada1p2    ONLINE       0     0     0

errors: No known data errors


In the end, the price really is not that much different for the hardware.




-- 
-------------------
Mike Tancsa, tel +1 519 651 3400
Sentex Communications, mike at sentex.net
Providing Internet services since 1994 www.sentex.net
Cambridge, Ontario Canada   http://www.tancsa.com/