gmultipath HA over iscsi/iser

[Max Gurtovoy]

Hi All,


On 6/19/2015 7:20 PM, John-Mark Gurney wrote:
> Max Gurtovoy wrote this message on Sun, Jun 14, 2015 at 19:16 +0300:
>> lately I was testing HA using gmultipath utility over iSCSI/iSER devices.
>> I'm working on 11-current code base.
>> I created 1 LUN on the target side and connected via 2 different
>> physical ports from the initiator side.
>> On the initiator side I see see /dev/da0 and /dev/da1.
>> I created multipath device using:
>> gmultipath label dm0 /dev/da0 /dev/da1.
>> Now I have new device /dev/multipath/dm0.
>> I set kern.iscsi.fail_on_disconnection=1 (to fail IO fast).
>>
>> Issue 1:
>> -------------
>> I can't run simple fio/dd traffice over /dev/da0 nor /dev/da1.
>> The only traffic that possible is using the multipath device dm0.
>> Is this by design ?
>
> This is probably because geom only lets one writer open the device at
> a time, and when the dm0 device is open, it blocks any other writers
> to it's underlying devices...  There is a special geom debug flag that
> you can set to avoid this...
>

I tried this debug exclusive flag with iSCSI/iSER but when I set it to 0 
the system crushed at the 'gmultipath label ...' stage.
John,
did you try it over iSCSI ?

>> In the linux implementation we can run traffic on both block devices and
>> multipath devices.
>>
>> Issue 2:
>> --------------
>> I run some fio traffic utility over multipath device dm0 on initiator
>> side with port toggling in a loop
>>
>> Port 1 down --> sleep 2 mins (iSCSI/ISER device reconnecting meanwhile
>> with no success) --> port 1 up --> sleep 5 mins (iSCSI/ISER device
>> reconnecting successecfully)
>> Port 2 down --> sleep 2 mins (iSCSI/ISER device reconnecting meanwhile
>> with no success) --> port 2 up --> sleep 5 mins (iSCSI/ISER device
>> reconnecting successecfully)
>>
>> The expected result is that when the port N is down than the traffic
>> moves to the available port and continue succesfully.
>> I run this test for many hours and traffic FAILED (even though there was
>> at least 1 suitable path between initiator and target).
>
> Someone else will need to answer this one...
>

I'm pretty new in the FreeBSD community so I don't know who is the 
maintainer of the gmultipath driver.
Maybe we can discuss about testing the gmultipath driver over iscsi/iser 
devices and fix some bugs together ?
We are planning to add it to our test plan and HA is in high priority 
for us.

Thanks,
Max