Another Adaptec 2120s perfomance problem

[kost]

Hi!

---PROBLEM---
I have got 2 servers running FBSD 5.x and RAID0.
The first server has RAID 0 with 2 IDE drives and the second one has
RAID0 with 8(!!!) SCSI drives.
All test I have run on these platforms gave me almost same results. My
testing was extremely simple and I expected to see at least 2 times
faster perfomance on SCSI RAID... but it didn't happened.

---HARDWARE---
server1: P4 3Ghz, Intel 875, 1Gb RAM, Promise + RAID 0(2xIDE Seagate 80Gb). OS: FreeBSD 5.1

server2: P4 3,2Ghz, Intel 865, 1Gb RAM, Adaptec 2120S(7349 firmware) + RAID 0(8x SCSI
Fujitsu 70Gb). OS FreeBSD(4.11, 5.2, 5.3, 5.4) and even Windows XP SP2
Corporate.

---TUNING---
1. Adaptec 2120s BIOS
First time I have left almost all settings in default state but then
I tried to turn off READ CACHE while creating ARRAY and turn on/off
WRITE cache.
2. FreeBSD
I'll show main FBSD settings for 5.3 RELEASE, because other versions had
same performance results.

sqlmail# uname -a
FreeBSD sqlmail.ropnet.ru 5.3-RELEASE FreeBSD 5.3-RELEASE #0:
Fri Apr 22 17:00:01 UTC 2005     root at sqlmail.ropnet.ru:/usr/src/sys/i386/compile/SQL  i386

sqlmail# df
Filesystem    1K-blocks   Used     Avail Capacity  Mounted on
/dev/aacd0s1a    988398  51620    857708     6%    /
devfs                 1      1         0   100%    /dev
/dev/aacd0s1d   2971278 735948   1997628    27%    /usr
/dev/aacd0s1f 550186622 260982 505910712     0%    /usr1
/dev/aacd0s1e    988398    510    908818     0%    /var

sqlmail# kldstat
Id Refs Address    Size     Name
 1    3 0xc0400000 368e5c   kernel
 2   14 0xc0769000 537f0    acpi.ko

Kernel was build in many variants: GENERIC, without aacp, without aacp
and pass device and etc.

When aacp was enabled I got following strange results:
sqlmail# dmesg
pass0: <FUJITSU MAT3073NP 0105> Fixed unknown SCSI-3 device
pass0: 160.000MB/s transfers (80.000MHz, offset 127, 16bit) 
pass1 at aacp0 bus 0 target 2 lun 0 
pass1: <FUJITSU MAT3073NP 0105> Fixed unknown SCSI-3 device 
pass1: 160.000MB/s transfers (80.000MHz, offset 127, 16bit) 
etc...

Why 160MB/s if Adaptec bios and aacli show 320MB/s? Is it normal? And
I saw nothing about "tagged queueing" enabled for at least one SCSI
device... and then my suspicion confirmed:

sqlmail# negotiate 0:1 -v
(pass0:aacp0:0:1:0): sync parameter: 8 
(pass0:aacp0:0:1:0): frequency: 160.000MHz 
(pass0:aacp0:0:1:0): offset: 127 
(pass0:aacp0:0:1:0): bus width: 16 bits 
(pass0:aacp0:0:1:0): disconnection is disabled 
(pass0:aacp0:0:1:0): tagged queueing is disabled   (!!!!!) 
aacp0: SIM/HBA version: 1 
aacp0: supports 16 bit wide SCSI 
aacp0: user has disabled initial BUS RESET or controller is in target/mixed mode    (!!!!!) 
aacp0: HBA engine count: 0 
aacp0: maximum target: 15 
aacp0: maximum LUN: 8 
aacp0: highest path ID in subsystem: 0 
aacp0: initiator ID: 7 
aacp0: SIM vendor: FreeBSD 
aacp0: HBA vendor: Adaptec 
aacp0: bus ID: 0 
aacp0: base transfer speed: 3.300MB/sec 

I didn't disabled BUS RESET as like didn't set controller in target/mixed
mode. I even don't now where to config these params, at least I
couldn't find anything in Adaptec BIOS on this theme.

Of course all my attempts to turn on "tagged queueing" was failed.


---TESTING AND RESULTS---
I understand that my tests are very far from ideal, but in my opinion
I have to see better performance with SCSI RAID even in such test,
especially we are talking not only about SCSI and IDE comparison but 2
vs 8(!) spindles performance.
Ok. what have I done:
sqlmail# cd /usr
sqlmail# fetch ftp://ftp.freebsd.org/pub/FreeBSD/ports/ports/ports.tar.gz
sqlmail# tar -zxf ports.tar.gz
sqlmail# time cp -pR ports.tar.gz /usr1/
0.577u 8.209s 2:12.44 6.6%      14+713k 41579+144io 75324pf+0w

While copying I run
sqlmail# iostat 1 100
on another console and got 3-5MB/sec speed for aacd0 device

sqlmail# systat -vmstat 1
showed me 90-100% disk busy while cp was in progress

So... results were almost the same in all kernel variants, all FBSD
versions and number of disks in SCSI RAID 0. And almost equal results
I got on server1 with IDE RAID 0.

Also I tried to build RAID5+0... It was awful, because it takes twice
more time to copy ports collection from one partition to another.

I tried to use another cable on 8 devices - same result
...tried to build ARRAY 0 with two SEAGATE CHEETAH - same result

Thank you for any answer.