NAS w/ multipath

[Gary Palmer]

On Tue, Feb 07, 2006 at 12:22:33PM -0800, Tom Samplonius wrote:
>   Now, in FreeBSD you could also multipath in the GEOM layer.  GEOM knows 
>   about devices going away, and knows how to handle that (ex. gmirror).  
> There is some support in GEOM for round-robin IO to two devices.  However, 
> phk has reported that the isp driver can hang forever on some timeouts, so 
> it might not be useful.  And I don't even know if GEOM round-robin is even 
> finished.

Be very careful with that kind of work.  There are SAN units out there
that do not share caches across multiple controllers.  To work around that
limitation, they have one controller than can "own" the LUN at any one
time, to ensure data consistency.  So if you write to a LUN through the
controller which does not "own" the LUN, it actually transfers "ownership"
of the LUN to the controller you sent the request through.  In at least
one implimentation, the LUN vanishes from both controllers for a period
measured in seconds while management of the LUN is handed off from one
controller to the other. The vendor worked around this with special
drivers that lived on the OS, which wasn't an option for us as they
didn't have FreeBSD support.

I suspect higher end devices (e.g. HDS and EMC Symmetrix units) this
isn't a problem, but in mid range and lower end stuff I'd expect problems
if the paths landed on separate controllers on the array.

Gary