configuration choices with Dell CERC (adaptec 2610SA)

[Scott Long]

user wrote:
> 
> On Sat, 26 Nov 2005, Herve Boulouis wrote:
> 
> 
>>Le 26/11/2005  14:17, user a �crit:
>>
>>>Read caching: yes/no (default is yes)
>>>
>>>(read caching does not rely on a battery, and is completely "safe",
>>>right?  I am going to set this to "yes", however I wonder - why would
>>>anyone _ever_ set it to "no" ?)
>>
>> 
>>You usually want to disable read caching on all aac based adaptec raid
>>controllers because it decreases performance.
> 
> 
> 
> Thanks - I did not know that, and that is good to know.
> 
> 
> 
>>>Write caching: enable/disable
>>>
>>>(this is the one that can get you into trouble if the system loses power
>>>before a write is committed to disk, and the battery is dead, right?  I am
>>>going to set this to "disable" because I do not think the 2610SA has a
>>>battery ... right ?)
>>
>>If your server is plugged on a UPS you can safely enable it, it's quite
>>a performance gain.
> 
> 
> 
> Ok.  It is.  However, my experience has been that no matter how rock solid
> the datacenter provided, UPS backed power is, there is a power
> interruption about every 1-2 years (and this experience comes from very
> very good datacenters - from Level3 to UUnet, etc.) ... so ... let's say I
> bank on this and enable write caching, how big of a deal is it when that
> power interruption does come ?
> 
> Assuming freebsd 6.0 and UFS2 ... is it just an fsck, or is there a
> potential for some real problems if the kernel thinks something is totally
> committed, but it never makes it to the drive ?
> 

The problem is not with filesystem corruption, it's with parity 
corruption.  It's possible to get a situation where a stripe is
partially written out and then interrupted by a power failure, leaving
the parity inconsistent with the stripe.  You won't notice this until
it's time to rebuild a failed drive from the parity, and then you'll
get corruption.

The battery on the card keeps power to the cache for a few hours until
the machine brought back online and the card can flush it.  Typically
the batteries are very overpriced, but I guess the extra insurance is
worth it and they don't wear out on machines that are kept constantly
on.  They wear out much faster in desktop machines that are turned off
quite a bit.

> 
> 
>>>Rebuild rate: high/medium/low
>>>
>>>(I have no idea what this means - I never saw this in adaptec bios before
>>>... can anyone define exactly what this means ?)
>>
>>This controls how much ressources the card will put in rebuilding a volume
>>that got a failed disk. A high rebuild rate will mean a short rebuild time
>>but slower disk accesses during the rebuild.
> 

Actually, what it controls is whether the firmware will insert a delay
in the rebuild task after every few i/o operations, in order to give
fairness to the i/o operations coming from the OS.  Last I checked, it
wasn't a terribly sophisticated algorithm.  Performance during a rebuild
is going to be horrible no matter what.  What is does have a clear
impact on is the performance of multiple rebuild tasks going on across
the same set of disks.  This isn't a terribly common occurrance, though.

> 
> 
> I am going to set it to "low" then ... my experience is that a failed disk
> often rebuilds itself only to re-fail shortly thereafter, thus making
> possible a continuous loop of rebuilding over and over ... which would
> essentially take the system offline if all resources were dedicated to the
> rebuild.  Comments ?

It would be better to let the rebuild happen as fast as possible.  Disks
have a bad habit of failing in groups, and you don't want to increase
the chance of a multi-disk failure by making recovery slow.

Scott