vinum performance

[Greg 'groggy' Lehey]

On Monday, 31 March 2003 at 22:01:25 -0500, Michael C. Brenner wrote:
> At 04:41 PM 3/31/2003, Jason Andresen wrote:
>>>>>> Ok. But I still don't understand why RAID 5 write performance is _so_
>>>>>> bad.
>>>>>> The CPU is not the bottle neck, it's rather bored. And I don't
>>>>>> understand
>>>>>> why RAID 0 doesn't give a big boost at all. Is the ahc driver known to
>>>>>> be
>>>>>> slow?
>>>>>
>>>>> (Both of these were on previously untouched files to prevent any
>>>>> caching, and the "write" test is on a new file, not rewriting an old
>>>>> one)
>> Write speed:
>> 81920000 bytes transferred in 3.761307 secs (21779663 bytes/sec)
>> Read speed:
>> 81920000 bytes transferred in 3.488978 secs (23479655 bytes/sec)
>>
>> But on the RAID5:
>> Write speed:
>> 81920000 bytes transferred in 17.651300 secs (4641018 bytes/sec)
>> Read speed:
>> 81920000 bytes transferred in 4.304083 secs (19033090 bytes/sec)
>
> Writing to a RAID5 stripe set requires that all disks in the array
> successfully report completion before the RAID5 controller's buffer can be
> released back to the cache.

No, you don't need to involve every disk, only the disk(s) to which
the data will be written, and the parity disk.

> (Applies to either software or hardware raid.)  If you are doing a
> large block write (like dd) you can easily fill the cache on most
> controllers.

You mean drive, right?  Most controllers don't have cache.

> Once the cache is full, the controller slows each write to the
> LONGEST completion time of each spindle in the array.

Yes, that's correct.

> ECC calculation becomes part of the latency also.

Well, yes, but most of the time it's negligible.  On modern machines,
we're talking in the order of few microseconds.

> In a 5 drive system (other than one where the cache is larger than
> the largest file being written as in a large EMC array) the writes
> are always about 4-5 times longer than the reads.

This is pretty much independent of the number of drives: assuming your
stripes aren't too small, you'll have two reads, two writes.  Yes,
they can go in parallel, but while they're doing it, they're blocking
other potential transfers.

Greg
--
See complete headers for address and phone numbers
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20030401/ea69e755/attachment.bin