Fibre Channel disks to two Systems?

[Jonathan Fosburgh]

On Friday 09 September 2005 01:59 pm, Danny Howard wrote:

>
> >From my research, I am thus far most impressed with the SANbloc 2Gb,
>
> which holds fourteen FC drives in a 3U rackmount.  It can be had with
> redundant RAID controllers, or as a JBOD.  There are similar products
> from other vendors as well.

Unfortunately I have no experience doing anything SAN related on FreeBSD, but 
I think I can say that you would do just fine with any Engenio-based  (IBM, 
StorageTek, Maxxan, etc) product.  They seem to manufacture most of the 
modular fibre channel storage out there, and they also sell FC-attached SATA 
hardware.  Of course if someone actually mentions they support FreeBSD then 
probably go with that, otherwise you are likely to be on your own.
>
> I could concievably do the RAID in software by running a gstripe across a
> set of gmirrors.
>
> As I understand it, I can have an FC loop with one or more drives,
> connected to two servers, and either server can talk to one or the other
> drives exclusively.  My QUESTION is: how is the arbitration done in
> FreeBSD?  You run camcontrol on either server and activate / deactivate
> drives in the loop?
>
> What happens if say, the primary server locks up in some weird manner?
> Can it block the backup server from talking to the drives?  (We can
> always have a NOC tech turn off a badly failed primary database, and
> power-cycle the disk array, if needed ...)
>
> A really far-out idea I had was that with fourteen drive bays I could
> have two hot spares, and then set up a stripe across four mirrored pairs
> (4x2 = 8-disk RAID10) and then with the remaining four drives assign
> each to be a third component of the gmirrored pairs, let the gmirrors
> sync up, then detach those drives from the gmirrors, mount them on the
> backup database, gstripe those containers together, and have a
> point-in-time "snapshot" of the drive array that could be mounted on the
> backup server, from which I could run database dumps, or conduct
> failover tests, etc.  (I could kick this around -geom. :)
>
> Uhmmm, has anyone done similar?  Suggestions?  Feedback?  Advice?
>
> Or, should I try to get a NetApp, or similar device, even though FreeBSD
> does not support iSCSI, because NFS performance over GigE may still beat
> FC?

I can't believe that FreeBSD would have such poor support for FC that NFS 
performs better. :)  Remember you dealing with block I/O on the storage array 
and file I/O on the NetApp (which I think supports either NFS or CIFS).  
There are (or have been) some proprietary database vendors (Microsoft comes 
to mind...) who don't allow their databases to run on any kind of NAS setup 
(which is what the NetApp is).  Granted I don't know if Postres gives support 
for putting databases on raw devices or if it only supports putting the 
database on a filesystem, so you may not get the full benefits of using block 
storage.  Still, for a database, I think you should use block storage if at 
all possible.  Alas, I can't speak to how you can give multiple FreeBSD 
systems access to the same storage device.
>
> Also, does anyone have a FreeBSD-friendly storage systems integrator or
> other vendor they can reccomend, particularly one near the San Francisco
> area?  I keep contacting various vendors who then fail to get back to
> me. :(

Personally I have never seen a storage vendor who supports FreeBSD, but I deal 
with large enterprise players where Linux is only just in the last couple of 
years enjoying broad support.  Maybe the smaller vendors are more likely to 
support FreeBSD.  Are there any Bay area FBSD user's groups? You might try 
them.

>
> Thanks for all feedback and suggestions!
>
> Sincerely,
> -danny

-- 
Jonathan Fosburgh
Storage Engineer/Architect