Tyan TA26, LSI 320-1, and FBSD6.0 Strangeness

[Ken Gunderson]

Hello List:

I'm having a tough time w/a Tyan TA26, 320-1 and 6.0-RELEASE that
I'm hoping y'all may be able to shed some light on.  I create logical
drives and install FBSD just fine.  Then cvsup, buildworld,
buildkernel, installkernel.  Upon reboot the system drives (mirror) are
in degraded mode and the raid0 drive (swap) is offline.  MegaRAID is
unable to rebuild the arrays.  I've called LSI support and they're
mystified as well.

Details:

A) Hardware:

1) Tyan Transport TA-26 2U Barebones Server

<http://www.tyan.com/products/html/ta26b2882.html>

Latest Tyan BIOS Rev. 3.04 b20

2) Dual AMD Opteron 246

3) 2 GB ECC Registered RAM (Corsair- Memtest runs ad infinitum w/o
reporting any errors.)

4) LSL MegaRAID 320-1 --Confirmed latest LSI Firmware

5) Fujitsu MAT3073NC 80 pin, 73.5 GB, U320-SCSI, 10K RPM Hard Drives

<http://www.fcpa.fujitsu.com/products/hard-drives/mat-3300-10k-rpm/specifications.html>


B) Configuration

1) Array 1 consists of 2 hard drives in slots 1 & 5 respecitvely w/2
logical drives configured:

a) Logical Drive 1- RAID1 Mirror, 64K stripe size.

b) Logical Drive 2- RAID0, 128K stripe size

2) Array 2

a) Logical Drive 3- RAID5 data array, 128K stripe size


3) Operating System- FreeBSD-6.0-RELEASE

4) Mainboard BIOS config- disable onboard Adaptec, SATA, etc.  Plug &
Play OS set to "NO"


C) How to Replicate:

1) create arrays, logical drives, and perform FreeBSD base install.

a) Logical Drive 1 system drive
b) Logical Drive 2- swap
c) Logical Drive 2- data storage

2) cvsup FreeBSD source code

3) buildworld, buildkernel, and installkernel

6) reboot

The logical drives 1 & 2 will now be offline and degraded mode and DMESG
reports IO errors.

Logical Drive 1- Degraded

Logical Drive 2- Offline

7) Boot into single user mode anyways, installworld, mergemaster, and
reboot.

8) Boot into LSI MegaRAID utilities

a) Objects>physical drives- drive in slot 1 will be failed
b) Try to rebuild- fails w/Error
c) Sometimes I get a partial rebuild before the error by going through
the rebuild menu.
d) No media errors, etc. reported.

At this point you're basically hosed and unable to recover.  My
understanding is that even if for some strange reason FBSD is
overwriting something on Physical Drive 1 that the MegaRAID should
still be able to rebuild from data on Physical Drive 5.


D) History:

1) Upon first encounter LSI tech support concluded that I had a
defective RAID card and rma'd.  Then same problems with new card.  

2) After much testing and discussion w/Tyan support conclusion was that
Slot 1 on the Tyan backplane was defective. Also possibly pci-riser as
it exhibited signs of physical damage. Tyan RMA's the machine.

3) Now I am stil having same problem.  Moving hd's around doesn't
help.  Not a HD issue as they work fine in another machine.


E) Present Status:

Interestingly enough, I am able to FORCE Physical Drive 1 back online
and then "Check Consistency".  Presently 21% complete so don't know if
it will choke on error on not yet.

LSI support is planning to investigate further but I am informed that
their test lab is under remodling at the moment and it won't be for at
least a couple weeks.

In the meantime any insights/help from the list would be appreciated.
All of this hardware is pretty well supported for some time now on
FBSD.  Am I on crack and missing the obvious here or what? 

TIA-- 

-- 
Best regards,

Ken Gunderson

Q: Because it reverses the logical flow of conversation.
A: Why is putting a reply at the top of the message frowned upon?