Tyan TA26, LSI 320-1, and FBSD6.0 Strangeness

[Scott Long]

Ken Gunderson wrote:

> On Tue, 20 Dec 2005 23:10:18 -0700
> Ken Gunderson <kgunders at teamcool.net> wrote:
> 
> 
>>On Tue, 20 Dec 2005 16:07:52 -0700
>>Ken Gunderson <kgunders at teamcool.net> wrote:
>>
>>
>>>Hello List:
>>>
>>>I'm having a tough time w/a Tyan TA26, 320-1 and 6.0-RELEASE that
>>>I'm hoping y'all may be able to shed some light on.  I create logical
>>>drives and install FBSD just fine.  Then cvsup, buildworld,
>>>buildkernel, installkernel.  Upon reboot the system drives (mirror) are
>>>in degraded mode and the raid0 drive (swap) is offline.  MegaRAID is
>>>unable to rebuild the arrays.  I've called LSI support and they're
>>>mystified as well.
>>
>>[big snippage]
>>
>>
>>>E) Present Status:
>>>
>>>Interestingly enough, I am able to FORCE Physical Drive 1 back online
>>>and then "Check Consistency".  Presently 21% complete so don't know if
>>>it will choke on error on not yet.
>>
>>Update- 
>>
>>The consistency check did complete w/o any errors and rebooting all
>>logical drives are once again in "Optimal" state.  For sake of
>>completeness heres the dmesg:
> 
> 
> [more snippage]
> 
> Yet another follow up on my own post...
> 
> Update Redoux:
> 
> 1) Using the amr driver from 7-CURRENT yields same results.  
> 
> 2) Did some testing playing musical hard drive slots.  IF I do NOT
> use slot 1 (# on Tyan Backplane starts w/1) and use the EXACT same raid
> config for the mirror usings, e.g. slots 2 & 3, then all works as
> normally expected.
> 
> So it would seem that Tyan and/or LSI have something Foobarred?  Or
> that for some reason FBSD is overwriting directly to disk on slot 1
> (i.e. da0) even though it's not technically there?
> 
> Bizarre hardware issues.  My raison d'etre...
> 

There is no way for FreeBSD to directly access disks attached to the
RAID controller.  All reads and writes to the array are bounded by the
controller, and there simply is no way to get around that.  With a
certain amount of advanced hacking it would be possible to corrupt the
disks with the amr_cam module, but even that is disabled with 7-CURRENT.
What I'd actually suspect is that the backplane and/or slot connector is
bad, so bad that simple parity detection cannot catch it.

Some controllers allow you to run scans on individual disks from within
a controlled environment, like the BIOS.  I don't recall if the LSI
cards have this feature, but if they do then they could almost certainly
verify this.

Scott