Dell H310, JBOD mode "hard error"

[Doug Ambrisko]

On Wed, Aug 14, 2013 at 04:16:23PM +0100, Steven Hartland wrote:
| 
| ----- Original Message ----- 
| From: "Doug Ambrisko" <ambrisko at ambrisko.com>
| To: <sbruno at freebsd.org>
| Cc: <FreeBSD-scsi at freebsd.org>
| Sent: Wednesday, August 14, 2013 4:03 PM
| Subject: Re: Dell H310, JBOD mode "hard error"
| 
| 
| >On Tue, Aug 13, 2013 at 07:46:56PM -0700, Sean Bruno wrote:
| >| On Sun, 2013-07-14 at 10:23 -0700, Sean Bruno wrote:
| >| > Not sure what to make of this.  I've tested a lot of svn revisions of
| >| > the thunderbolt code, but nothing looks obvious.  
| >| > 
| >| > When I use a single drive in "SYSPD" mode on a Dell H310 (falcon or
| >| > skinny drake) I get a /dev/mfisyspd0 device.  The JBOD mode *seems* to
| >| > work just fine as long as I don't do multiple things at once to it, 
| >e.g.
| >| > single user fsck works, but multiuser things die.
| >| > 
| >| > I get a failure case that emits errors such as:
| >| > 
| >| > g_vfs_done():error 27 in callback
| >| > mfisyspd0p2[READ(offset=7176192, length=425984)]mfisyspd0: hard error
| >| > error = 5
| >| > cmd=read 15360-16383
| >| > error 27 in callback
| >| > g_vfs_done():mfisyspd0: hard error mfisyspd0p2[READ(offset=7602176,
| >| > length=524288)]cmd=read error = 5
| >| > 16384-17407
| >| > error 27 in callback
| >| > g_vfs_done():mfisyspd0: hard error mfisyspd0p2[READ(offset=8126464,
| >| > length=524288)]cmd=read error = 5
| >| > 14560-15359
| >| > error 27 in callback
| >| > g_vfs_done():mfisyspd0: hard error mfisyspd0p2[READ(offset=7192576,
| >| > length=409600)]cmd=read error = 5
| >| > 15360-16383
| >| > 
| >| > 
| >| > Sean
| >| 
| >| Ah, I see something that Yahoo! does that FreeBSD does not finally.  We
| >| tune MAXPHYS *up* to (512 * 1024) because of performance and available
| >| memory.
| >| 
| >| mfi(4) set's its own (MFI_MAXPHYS) to (128 * 1024) instead of using the
| >| value from sys/param.h  (btw, I don't quite get why, but whatever).
| >| 
| >| Without a min() check in mfi_syspd.c that mimics the one in mfi_disk.c,
| >| Yahoo code falls over in "SYSPD" mode (mfi(4) real jbod mode).
| >| 
| >| Patch:
| >| 
| >| Index: mfi_syspd.c
| >| ===================================================================
| >| --- mfi_syspd.c (revision 254313)
| >| +++ mfi_syspd.c (working copy)
| >| @@ -128,7 +128,9 @@
| >|  sc->pd_disk->d_drv1 = sc;
| >|  sc->pd_disk->d_maxsize = sc->pd_controller->mfi_max_io * secsize;
| >|  sc->pd_disk->d_name = "mfisyspd";
| >| - sc->pd_disk->d_open = mfi_syspd_open;
| >| + sc->pd_disk->d_maxsize = min(sc->pd_controller->mfi_max_io * secsize,
| >| + (sc->pd_controller->mfi_max_sge - 1) * PAGE_SIZE);
| >| +
| >|  sc->pd_disk->d_close = mfi_syspd_close;
| >|  sc->pd_disk->d_strategy = mfi_syspd_strategy;
| >|  sc->pd_disk->d_dump = mfi_syspd_dump;
| >
| >
| >That change for d_maxsize looks okay but do you really want to get
| >rid of d_open?  I assume this is a cut-n-paste type error and the patch
| >(hand editted) should be:
| >
| >Index: mfi_syspd.c
| >===================================================================
| >--- mfi_syspd.c (revision 254313)
| >+++ mfi_syspd.c (working copy)
| >@@ -128,7 +128,8 @@
| > sc->pd_disk->d_drv1 = sc;
| >- sc->pd_disk->d_maxsize = sc->pd_controller->mfi_max_io * secsize;
| >+ sc->pd_disk->d_maxsize = min(sc->pd_controller->mfi_max_io * secsize,
| >+ (sc->pd_controller->mfi_max_sge - 1) * PAGE_SIZE);
| > sc->pd_disk->d_name = "mfisyspd";
| >sc->pd_disk->d_open = mfi_syspd_open;
| > sc->pd_disk->d_close = mfi_syspd_close;
| > sc->pd_disk->d_strategy = mfi_syspd_strategy;
| > sc->pd_disk->d_dump = mfi_syspd_dump;
| >
| >BTW, has mfiutil been updated to create real JBODs versus the RAID per
| >drive?  I know someone was talking about doing that.  A note on 
| >implementing
| >it, it also requires JBOD to be enabled in the controller.  I'm not sure
| >if all controllers support it.  I forget when I was playing with it.  I've
| >always wondering if we should change the name for the syspd disk node
| >but I left it for compatibility with LSI.  We could do an alias.  It is
| >good in away that it doesn't create /dev/mfid* nodes so that it is easier
| >to track bugs.
| 
| Some do support native JBOD, some require an additional controller level
| setting, and some simply don't support it and I'm not aware of an easy way
| to determine which is the case I'm afraid.

It can be figured out via MegaCli.  I remember MegaaCli can set it.
I think it can find out if the controller supports it.  MegaCli was
more powerful then the POST tool since with I was able to have both
JBODs and RAID volumes on the same controller.  I recall some of the
POST tools prevented that.  I also need to play around with CacheCade
more and how to set that up.  Again, I was able to do via MegaCli
except for the license activation.  However, at the time with the
version of CacheCade that was in the firmware I was using it slowed
things down versus speeding it up.  That should be fixed now but I
haven't had time to test it out.  I should finally have time to work
on more FreeBSD things since some other issues I've been working on
have been resolved.
 
| Last time I looked mfiutil used RAID not native JBOD for JBOD's

That's probably the case.

Thanks,

Doug A.