3ware 9500 twa driver support

[Phillip Steinbachs]

Greetings,

I've been following the threads relating to 3ware 8506 and 9500 series
cards under the amd64 port.  I ran into stability problems with the
8506-8 as reported by others here and 3ware themselves (PCI bus timing
issues in conjunction with Tyan riser cards).  I replaced the 8506-8 with
a 9500S-8, and I think the hardware stability issues are fixed.  But there
is the software driver issue to contend with now.  First off, here are the
system specs:

Motherboard:  Tyan Thunder K8S Pro (S2882)
   BIOS Rev:  V2.02 (06/01/04)
       CPUs:  2 x AMD Opteron 246 (2GHz)
        RAM:  4 x Crucial CT6472Y40B (PC3200 DDR400, 512MB ECC Reg.)
	      	- 2 x 512MB DIMMs per CPU
System Disk:  2 x Maxtor 6Y120MO (120GB SATA)
  Data Disk:  1 x 3ware 9500S-8
		- 4 x Maxtor 6Y250M0 (250GB SATA) as HW RAID5

         OS:  FreeBSD 5.2.1-RELEASE-p8 (amd64)
 TWA driver:  twa.tgz 5.2 code (release 9.0.1) from 3ware.com

Relevant BIOS configuration:

  - ACPI is enabled (machine consistently reboots right before FreeBSD
    boot loader if it's not turned on)
  - PnP is enabled (tried on and off with no noticable differences)
  - Northbridge is set for node interleave, ECC scrubbing is disabled

Relevant OS configuration:

120GB SATA system disks as RAID 1 mirror via atacontrol

3ware twa driver problem:

I have successfully compiled the twa driver from 3ware's source, and the
system recognizes the card properly.  The da0 device is accessible for
fdisk and label.  Everything looks ok until it's time for newfs to create
a filesystem.  Basically, newfs starts and doesn't get anywhere.  The
system doesn't hang (unlike other reports I've read), but the fs creation
never finishes (even after several hours).  I tried breaking the space up
into smaller filesystems, but none of those would complete either.  I have
TWA_DEBUG set to 5, but there are no errors being reported anywhere.

Now, I recognize this configuration isn't supported because 3ware hasn't
certified their driver on amd64 yet.  The big question is, what is wrong
with it, and what if any configuration exists that does work (even if it's
not yet officially supported)?

It would appear the options are:

  - Try twa from 5-2.CURRENT under 5.2.1-RELEASE-p8 (reportedly this has
    been tried, with compilation failures) (amd64)

  - Use FreeBSD 5.2-CURRENT (amd64)

  - Use FreeBSD 5.2.1-RELEASE-p8 (i386) with 3ware's twa driver
    (supported)


I have tried the last two options:

FreeBSD 5.2-CURRENT compiles fine, but I've run into the getrootbyname
problem with ata/SATA.  Thus, I've been unable to test if the twa driver
works and is stable in that case.  Unless I'm mistaken, this is still a
problem with 5.2-CURRENT.  I'm wary of using 5.2-CURRENT in production,
as I should be, but am willing to test fixes if it means it will lead to
stability in the relative near future.  There was supposedly a patch
around March to revert the code that broke this, but it didn't apply
cleany to a snapshot from a few days ago.

Trying to boot the FreeBSD-5.2.1-RELEASE (i386) installer CD results in a
consistent panic/reboot before drives are detected (this same problem has
been reported by others).  I will post the dmesg output if it would be
helpful... the box isn't available for me to do so at the moment.   I've
tried disabling ACPI in the BIOS and from the boot loader, and various
other BIOS configuration changes to no avail.  I would be ok running in
32bit mode for the time being, but it seems more quirky than amd64 at
this point, on this specific hardware.

As a final note, I'm evaluating FreeBSD because Solaris x86 doesn't have
[any|sufficient] hardware IDE-RAID support at the current time.  If it
did, Solaris would be my preference because I've run it for many years
with great success.  I've used FreeBSD 4.x for smaller machines (i.e
firewalls) and found it to be quite stable and functional.  However, my
load testing with 5.2.1 on this new hardware has been very promising,
aside from the problems I'm reporting. If anyone has suggestions
regarding the most stable software configuration for this hardware, it
would be most appreciated.

Thanks.

-phillip

P.S.  For what it's worth, this hardware will replace a machine that is
now hosting:

windowmaker.org
dockapps.org
irc.sage-members.org (openirc.net)

With the new hardware, I will finally be able to setup;

usebsd.org (high-speed *BSD mirror site, with multi-homed connectivity
through UUnet, Level3, Cable & Wireless, and SprintLink)

I think it would be fitting if this site was powered by FreeBSD, eh?