Serious Dell Sadness - H200, H700, and H800

[Tom Daly]

Hey Folks,
First, happy new year! Thanks for all of your ongoing support of FreeBSD, which we love at DynDNS.

I wanted to drop a note to this list, and give people some thoughts on some recent troubles we've been having with FreeBSD 7.3 and 8.1, and Dell's newest 11th generation servers. I'd say these servers are pretty robust overall: the chassis is well built, there is good air flow, and the rack system is well thought out. From a CPU/RAM/Network I/O perspective, they run fast.

However, we've been having serious ongoing problems with the Dell RAID controllers in the 11th gen series. The first card we tried, the H200, is supported by the MPS driver in HEAD. We compiled the driver into an 8.1 kernel, and we're able to detect disks configured as a JBOD. When we configured 2x 1TB disks into a RAID 1 volume, FreeBSD and the MPS driver didn't detect the disks.

We moved on to the H700 on R510 loaded with 12x 15k RPM SAS disks in a RAID 10 with an approx. RAID volume size of 3.6TB. FreeBSD is able to see these cards and drives, in various RAID configurations with both FreeBSD 8.x and 7.x. We're able to install FreeBSD and boot up, but trouble starts to occur when we put load on the arrays. Under a bonnie++ test on FreeBSD 8.1, we're seeing regular timeouts on the mfi device, where the disks just "disappear" for up to 1 hour, then just come back. We cannot reproduce the same failure on FreeBSD 7.3, but we see our disk I/O performance drop by approx. 33%.

We found this post (https://forums.freebsd.org/showthread.php?p=118346) and applied the latest firmware patch to the controller (http://ftp.us.dell.com/SAS-RAID/DELL_PERC-H800-ADAPTER_A06_R285986.txt)

We appear to be seeing the same behavior with the H800 and external arrays.

We're getting ready to run a battery of tests with the same large RAID volumes with Debian and Dell's drivers, and bonnie++. We're trying to establish a hardware problem or a software problem. We suspect that volumes of > 2TB have something to do with this, so we'll be stress testing with some smaller volume sizes. Does anyone have any thoughts on this?

I'd love to hear any experiences people are having with Dell H200, H700, and H800 cards. Hopefully the list finds this information helpful.

Regards,
Tom

-- 
Tom Daly
CTO, Dynamic Network Services, Inc.
http://dyn.com/