SMP causes weird performance problems

[Markus Kovero]

Hi everyone, basically I've encountered very strange (for me atleast) 
performance problem while tuning Dell Poweredge 1800.
I got 4-disk RAID5 set with harware controller made by Adaptec (works 
with aac) and everything is fine until I recompile kernel with SMP support.
There are 2x Xeon CPU's with hyperthreading enabled (tried htt disabled 
too) but after SMP turned on disk reads are affected somehow and reads 
drop 320-350MB/s  (with small files <1GB) to 110-150MB/s.
Disk writes slow down 1-2MB/s but thats acceptable for me.

Mainly I played around with dd but investigated it with bonnie++ too and 
noticed same behaviour.

After that, removed RAID5 array, tried RAID10 (no win) and then even  
with single volumes. No help, disk reads slow down dramatically after 
turning SMP on.

There's no problem whatsoever SMP turned on or not in linux, reads and 
writes are snappy. I just don't want linux. Ever. Again. (guys easily 
drawn into flame wars dont read that last one:)

I've tried FreeBSD 5.4 and 6.0_BETA5, with and without adaptive_giant (I 
read it may cause great slowdowns).

Any ideas, suggestions..?

Yours
Markus Kovero
(mui)