mfi (Dell H700) + hot swapping doesn't appear to work with RC1

[John Baldwin]

On Thursday, December 15, 2011 4:19:58 am Jan Mikkelsen wrote:
> On 15/12/2011, at 2:16 AM, Borja Marcos wrote:
> 
> > 
> > On Dec 14, 2011, at 2:09 PM, Hugo Silva wrote:
> > 
> >> Hello,
> >> 
> >> First of all apologies if this has been fixed in RC3. I set this server
> >> up with mfsbsd, which is RC1, and didn't get to update the system yet.
> >> 
> >> This box has 6 hdds, a 2-mirror zpool was set up as the root pool, with
> >> 2 spares.
> >> 
> >> While testing hot swapping I noticed that while the controller detects
> >> disk removal/insertion, the zpool will never recover. The problem seems
> >> to be deeper than ZFS, as disklabel/fdisk/etc also fail on the
> >> removed-and-reinserted disk.
> >> 
> >> At the ZFS level, doing a zpool clear yields more errors on the removed
> >> disk; rebooting becomes the only option to make the pool healthy again.
> >> 
> >> 
> >> Is this normal? Did I miss any step?
> > 
> > I assume that you have tried to use the H700 as a "JBOD" card, defining 
logical volume for each hard disk.
> > 
> > The problem is: that gorgeous, fantastic, masterful, Nobel award candidate 
card, has a wonderful behavior in that case. If you extract one of the disks, 
the logical volume associated to it is invalidated. So, you insert a 
replacement disk, and the card refuses to recognize the volume. What is even 
worse, in order to recover it's mandatory to reboot the complete system *AND* 
go through the RAID configuration utility.
> > 
> > That's the problem. The card refuses  to work as a simple disk controller 
without frills, and the frills get in the way.
> > 
> > To summarize: it isn't FreeBSD's fault, no matter which version you use. 
It's a "feature" coming directly from the geniuses who designed the card.
> 
> (Sending again to avoid moderation.)
> 
> Hugo: You missed a step. Borja: No reboot required.
> 
> For the mfi controllers I have been testing recently (MegaRAID 9261-8i), you 
need to install the sysutils/megacli port, and use that to clear the 
"foreignness" of the disk you just added. Something like:
> 
>    MegaCli -CfgForeign -Clear -a0
> 
> You should be able to then recreate it as a JBOD device, and progress 
through whatever higher level recovery you need to do.

Can you do this by marking it as 'good' via mfiutil and then using mfiutil
to create a volume?

-- 
John Baldwin