large RAID volume partition strategy

[Vivek Khera]

I have a shiny new big RAID array.  16x500GB SATA 300+NCQ drives  
connected to the host via 4Gb fibre channel.  This gives me 6.5Tb of  
raw disk.

I've come up with three possibilities on organizing this disk.  My  
needs are really for a single 1Tb file system on which I will run  
postgres.  However, in the future I'm not sure what I'll really need.  
I don't plan to ever connect any other servers to this RAID unit.

The three choices I've come with so far are:

1) Make one RAID volume of 6.5Tb (in a RAID6 + hot spare  
configuration), and make one FreeBSD file system on the whole partition.

2) Make one RAID volume of 6.5Tb (in a RAID6 + hot spare  
configuration), and make 6 FreeBSD partitions with one file system each.

3) Make 6 RAID volumes and expose them to FreeBSD as multiple drives,  
then make one partition + file system on each "disk".  Each RAID  
volume would span across all 16 drives, and I could make the volumes  
of differing RAID levels, if needed, but I'd probably stick with RAID6 
+spare.

I'm not keen on option 1 because of the potentially long fsck times  
after a crash.

What advantage/disadvantage would I have between 2 and 3?  The only  
thing I can come up with is that the disk scheduling algorithm in  
FreeBSD might not be optimal if the drives really are not truly  
independent as they are really backed by the same 16 drives, so  
option 2 might be better.  However, with option 3, if I do ever end  
up connecting another host to the array, I can assign some of the  
volumes to the other host(s).

My goal is speed, speed, speed.  I'm running FreeBSD 6.2/amd64 and  
using an LSI fibre card.

Thanks for any opinions and recommendations.


=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Vivek Khera, Ph.D.                Khera Communications, Inc.
Internet: khera at kciLink.com       Rockville, MD  +1-301-869-4449 x806