nforce4 disk I/O

[Dieter]

I thought I tested disk I/O in the past and got full write
speed to 4 drives at once, but now I get a slowdown if
writing to more than one disk at once.

FreeBSD 6.0
FFS with soft updates

nvidia nforce 4 ultra
4 SATA drives
1 PATA drive
1 PATA CD/DVD (idle)

Doing dd from /dev/zero to 1 drive,  I get about   7 MB/s.
Doing dd from /dev/zero to 2 drives, I get about   8 MB/s total.
Doing dd from /dev/zero to 3 drives, I get about 8-9 MB/s total.

Reading from one drive doesn't hurt the read or write speed to a second drive.

I'm also getting things like one process that is I/O bound writing to a
disk will significantly slow down a CPU bound process writing to a different
disk, and the CPU becomes over 50% idle.

I wonder if there is contention for a lock that only happens when writing?
Does the driver hold a lock while waiting for the disk to say "yes I wrote
that sector" ?


atapci0: <nVidia nForce4 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xe000-0xe00f at device 6.0 on pci0
ata0: <ATA channel 0> on atapci0
ata1: <ATA channel 1> on atapci0
atapci1: <nVidia nForce4 SATA150 controller> port 0x9f0-0x9f7,0xbf0-0xbf3,0x970-0x977,0xb70-0xb73,0xcc00-0xcc0f mem 0xfebfb000-0xfebfbfff irq 10 at device 7.0 on pci0
ata2: <ATA channel 0> on atapci1
ata3: <ATA channel 1> on atapci1
atapci2: <nVidia nForce4 SATA150 controller> port 0x9e0-0x9e7,0xbe0-0xbe3,0x960-0x967,0xb60-0xb63,0xb800-0xb80f mem 0xfebfa000-0xfebfafff irq 11 at device 8.0 on pci0
ata4: <ATA channel 0> on atapci2
ata5: <ATA channel 1> on atapci2
acd0: DVDR <HL-DT-ST DVDRAM GSA-4160B/A301> at ata0-master UDMA66
ad2: 305245MB <Seagate ST3320620A 3.AAC> at ata1-master UDMA100
ad4: 238475MB <Seagate ST3250823AS 3.03> at ata2-master SATA150
ad6: 238475MB <Seagate ST3250823AS 3.03> at ata3-master SATA150
ad8: 238475MB <Seagate ST3250823AS 3.03> at ata4-master SATA150
ad10: 305245MB <Seagate ST3320620AS 3.AAC> at ata5-master SATA150
cd0 at ata0 bus 0 target 0 lun 0