Request for opinions - gvinum or ccd?

[Freddie Cash]

On Sun, Jun 7, 2009 at 12:01 AM, Wojciech Puchar<
wojtek at wojtek.tensor.gdynia.pl> wrote:
>>> (very roughly, in the non-sequential access case) expected to deliver
>>> performance of four drives in a RAID0 array?
>>
>> According to all the Sun documentation, the I/O throughput of a raidz
>> configuration is equal to that of a single drive.
>
> exactly what i say. it's like RAID3. Not RAID5 which have close to n times
> single drive throughput on read and rougly n/4 on writes.
>
>> We remade the pool using 3x 8-drive raidz2 vdevs, and performance has
>> been great (400 MBytes/s write, almost 3 GBytes/s sequential read, 800
>
> why write performance is so slow? in Sun theory it should have the same
> speed as reads. I would say that it should be even better a bit -
filesystem
> get data first in cache and can plan ahead.
>
>> MBytes/s random read).
>
> random read on how big chunks?
>
> Are you sure you get 3GB/s on read? it would mean each drive must be able
to
> do 140MB/s
>
> What disks do you use?

12x 500 GB Seagate EL2 SATA drives, part of their enterprise near-line
storage line.
12x 500 GB WD SATA drives, generic off-the-shelf drives

I re-ran the iozone tests, letting them run to completion, and here are the
results:

The iozone command:  iozone -M -e -+u -T -t <threads> -r 128k -s 40960 -i 0
-i 1 -i 2 -i 8 -+p 70 -C
I ran the command using 32, 64, 128, and 256 for <threads>

Write speeds range from 236 MBytes/sec to 582 MBytes/sec for sequential; and
from 242 MBytes/sec to 550 MBytes/sec for random.

Read speeds range from 3.3 GBytes/sec to 5.5 GBytes/sec for sequential; and
from 1.8 GBytes/sec to 5.5 GBytes/sec for random.

All the gory details are below.

32-threads:  Children see ...  32 initial writers =  582468.13 KB/sec
32-threads:  Parent sees  ...  32 initial writers =  108808.46 KB/sec
64-threads:  Children see ...  64 initial writers =  236144.47 KB/sec
64-threads:  Parent sees  ...  64 initial writers =   86942.94 KB/sec
128-threads: Children see ... 128 initial writers =  284706.68 KB/sec
128-threads: Parent sees  ... 128 initial writers =   10850.40 KB/sec
256-threads: Children see ... 256 initial writers =  258260.59 KB/sec
256-threads: Parent sees  ... 256 initial writers =    9882.16 KB/sec

32-threads:  Children see ...  32 rewriters =  545347.52 KB/sec
32-threads:  Parent sees  ...  32 rewriters =  339308.08 KB/sec
64-threads:  Children see ...  64 rewriters =  419838.51 KB/sec
64-threads:  Parent sees  ...  64 rewriters =  335620.45 KB/sec
128-threads: Children see ... 128 rewriters =  350668.51 KB/sec
128-threads: Parent sees  ... 128 rewriters =  319452.97 KB/sec
256-threads: Children see ... 256 rewriters =  317751.52 KB/sec
256-threads: Parent sees  ... 256 rewriters =  295579.66 KB/sec

32-threads:  Children see ...  32 random writers =  379256.37 KB/sec
32-threads:  Parent sees  ...  32 random writers =   95298.44 KB/sec
64-threads:  Children see ...  64 random writers =  551767.68 KB/sec
64-threads:  Parent sees  ...  64 random writers =  113397.95 KB/sec
128-threads: Children see ... 128 random writers =  241980.60 KB/sec
128-threads: Parent sees  ... 128 random writers =   74584.01 KB/sec
256-threads: Children see ... 256 random writers =  398427.84 KB/sec
256-threads: Parent sees  ... 256 random writers =   20219.56 KB/sec

32-threads:  Children see ...  32 readers = 5023742.86 KB/sec
32-threads:  Parent sees  ...  32 readers = 4661309.72 KB/sec
64-threads:  Children see ...  64 readers = 5516460.71 KB/sec
64-threads:  Parent sees  ...  64 readers = 3949337.61 KB/sec
128-threads: Children see ... 128 readers = 4748635.74 KB/sec
128-threads: Parent sees  ... 128 readers = 3208982.03 KB/sec
256-threads: Children see ... 256 readers = 4358453.38 KB/sec
256-threads: Parent sees  ... 256 readers = 2741593.08 KB/sec

32-threads:  Children see ...  32 re-readers = 5502926.62 KB/sec
32-threads:  Parent sees  ...  32 re-readers = 4650327.75 KB/sec
64-threads:  Children see ...  64 re-readers = 5509400.02 KB/sec
64-threads:  Parent sees  ...  64 re-readers = 4526444.40 KB/sec
128-threads: Children see ... 128 re-readers = 4072363.55 KB/sec
128-threads: Parent sees  ... 128 re-readers = 2840317.47 KB/sec
256-threads: Children see ... 256 re-readers = 3329375.95 KB/sec
256-threads: Parent sees  ... 256 re-readers = 2183894.33 KB/sec

32-threads:  Children see ...  32 random readers = 5555090.45 KB/sec
32-threads:  Parent sees  ...  32 random readers = 4602383.62 KB/sec
64-threads:  Children see ...  64 random readers = 4402270.77 KB/sec
64-threads:  Parent sees  ...  64 random readers = 2059081.52 KB/sec
128-threads: Children see ... 128 random readers = 3070466.93 KB/sec
128-threads: Parent sees  ... 128 random readers =  525076.11 KB/sec
256-threads: Children see ... 256 random readers = 1888676.12 KB/sec
256-threads: Parent sees  ... 256 random readers =  293304.53 KB/sec

32-threads:  Children see ...  32 mixed workload = 3130000.18 KB/sec
32-threads:  Parent sees  ...  32 mixed workload =  123281.78 KB/sec
64-threads:  Children see ...  64 mixed workload = 1587053.33 KB/sec
64-threads:  Parent sees  ...  64 mixed workload =  294586.82 KB/sec
128-threads: Children see ... 128 mixed workload =  807349.95 KB/sec
128-threads: Parent sees  ... 128 mixed workload =   98998.77 KB/sec
256-threads: Children see ... 256 mixed workload =  393469.55 KB/sec
256-threads: Parent sees  ... 256 mixed workload =  112394.90 KB/sec

-- 
Freddie Cash
fjwcash at gmail.com