performance problem with gstripe

[Joel Jacobson]

here's what i did:

-------------<snip>--------------

ca-sbox-2# foreach i (0 1)
foreach? dd if=/dev/zero of=/dev/da$i bs=512k count=1024 &
foreach? end
[1] 5402
[2] 5403
ca-sbox-2# 1024+0 records in
1024+0 records out
536870912 bytes transferred in 4.262723 secs (125945532 bytes/sec)
1024+0 records in
1024+0 records out
536870912 bytes transferred in 4.272499 secs (125657357 bytes/sec)

ca-sbox-2# gstripe create -s 262144 d0 /dev/da{0,1}
ca-sbox-2# dd if=/dev/zero of=/dev/stripe/d0 bs=512k count=4096
4096+0 records in
4096+0 records out
2147483648 bytes transferred in 34.124683 secs (62930508 bytes/sec)

ca-sbox-2# newfs /dev/stripe/d0 > /dev/null
ca-sbox-2# mount /dev/stripe/d0 /mnt
ca-sbox-2# dd if=/dev/zero of=/mnt/bigfile bs=512k count=4096 && /usr/ 
bin/time sync
4096+0 records in
4096+0 records out
2147483648 bytes transferred in 11.081184 secs (193795502 bytes/sec)
        0.06 real         0.00 user         0.04 sys

# sysctl kern.geom
kern.geom.collectstats: 1
kern.geom.debugflags: 0
kern.geom.label.debug: 0
kern.geom.stripe.fast_failed: 0
kern.geom.stripe.maxmem: 13107200
kern.geom.stripe.fast: 1
kern.geom.stripe.debug: 0

-----------<snip>-----------

- j

On Jan 6, 2009, at 7:05 PM, Ivan Voras wrote:

> Joel Jacobson wrote:
>> i have a bit of a weird issue, which i suspect is a configuration
>> problem, and was looking for a little advice.  i have an LSI JBOD box
>> with a bunch of SAS drives that i would like to gstripe together.   
>> each
>> drive individually seems to be able to do about 80 MB/sec streaming
>> write, and doing parallel dd's gives me the 160 MB/sec i would  
>> expect.
>> if i gstripe them together with a 256k stripe width, i only see 80
>> MB/sec, though.
>
> How do you measure this? If with dd, what block size (bs) do you use?
>
>>
>> if, however, i newfs/mount it as ufs and then dd myself a big file,  
>> that
>> gets me about 120-130 MB/sec.
>>
>> why does mounting matter?
>
> I'd guess because of write caching that enables you to make use of
> multiple stripes at once, if your dd bs is smaller than 2*stripe size.
>