gmultipath HA over iscsi/iser

[John-Mark Gurney]

Max Gurtovoy wrote this message on Tue, Jun 23, 2015 at 18:42 +0300:
> On 6/19/2015 7:20 PM, John-Mark Gurney wrote:
> > Max Gurtovoy wrote this message on Sun, Jun 14, 2015 at 19:16 +0300:
> >> lately I was testing HA using gmultipath utility over iSCSI/iSER devices.
> >> I'm working on 11-current code base.
> >> I created 1 LUN on the target side and connected via 2 different
> >> physical ports from the initiator side.
> >> On the initiator side I see see /dev/da0 and /dev/da1.
> >> I created multipath device using:
> >> gmultipath label dm0 /dev/da0 /dev/da1.
> >> Now I have new device /dev/multipath/dm0.
> >> I set kern.iscsi.fail_on_disconnection=1 (to fail IO fast).
> >>
> >> Issue 1:
> >> -------------
> >> I can't run simple fio/dd traffice over /dev/da0 nor /dev/da1.
> >> The only traffic that possible is using the multipath device dm0.
> >> Is this by design ?
> >
> > This is probably because geom only lets one writer open the device at
> > a time, and when the dm0 device is open, it blocks any other writers
> > to it's underlying devices...  There is a special geom debug flag that
> > you can set to avoid this...
> 
> I tried this debug exclusive flag with iSCSI/iSER but when I set it to 0 
> the system crushed at the 'gmultipath label ...' stage.
> John,
> did you try it over iSCSI ?

Nope, I didn't..  In fact, I was thinking of another debug flag the
one in kern.geom.debugflags, but that one should have worked too...
Since it doesn't, please file a bug report and notify the maintainer of
multipath with a complete report...

> >> In the linux implementation we can run traffic on both block devices and
> >> multipath devices.
> >>
> >> Issue 2:
> >> --------------
> >> I run some fio traffic utility over multipath device dm0 on initiator
> >> side with port toggling in a loop
> >>
> >> Port 1 down --> sleep 2 mins (iSCSI/ISER device reconnecting meanwhile
> >> with no success) --> port 1 up --> sleep 5 mins (iSCSI/ISER device
> >> reconnecting successecfully)
> >> Port 2 down --> sleep 2 mins (iSCSI/ISER device reconnecting meanwhile
> >> with no success) --> port 2 up --> sleep 5 mins (iSCSI/ISER device
> >> reconnecting successecfully)
> >>
> >> The expected result is that when the port N is down than the traffic
> >> moves to the available port and continue succesfully.
> >> I run this test for many hours and traffic FAILED (even though there was
> >> at least 1 suitable path between initiator and target).
> >
> > Someone else will need to answer this one...
> 
> I'm pretty new in the FreeBSD community so I don't know who is the 
> maintainer of the gmultipath driver.

Though I realize it's difficult, it's easiest to look at the source to
see who's been touching it last:
https://svnweb.freebsd.org/base/head/sys/geom/multipath/g_multipath.c?view=log

Looks like mav has been somewhat active recently...  I've cc'd him...

> Maybe we can discuss about testing the gmultipath driver over iscsi/iser 
> devices and fix some bugs together ?

I can help out some, but engaging mav, or some of the others that
are more active in storage would be better...

> We are planning to add it to our test plan and HA is in high priority 
> for us.

You should definately talk/coordinate w/ the people at iXsystems (mav
works there), as they work in storage w/ iSCSI, etc.

-- 
  John-Mark Gurney				Voice: +1 415 225 5579

     "All that I will do, has been done, All that I have, has not."