How to get fibre channel multipath working (FreeBSD target, mac initiator)

[Aram Hăvărneanu]

Hello,

(I hope this is the correct mailing list for these type of things).

I am using FreeBSD as a fibre channel target. The initiator is a
mac, connected with a point to point connection. I wish to use
multipathing. I have a QLogic QLE2462 card, and I have recompiled
the kernel to enable target mode support.

It works, but I can't get multipathing to work. The mac sees two
LUNs and connects to both.

My understanding is that multipathing should "just work" on the mac
side, if what the mac calls "Logical Unit Unique ID" is the same
for every path. I believe that the "Logical Unit Unique ID" is the
SCSI INQUIRY VPD page 0x80 data. So option "-S" in ctladm(8). But
perhaps I am mistaken and it is the device id, so option "-d" in
ctladm(8).  To cover both cases, I have this /etc/ctl.conf
configuration:

    lun fibre-channel {
            backend block
            device-id "FC Disk AAAA0000ZZZ01"
            serial "AAAA0000ZZZ01"
            option vendor "MGK"
            option product "FC Disk"
            path /dev/zvol/tank1/export/fc
    }

    target naa.2100001b321cd5b6 {
            auth-group no-authentication
            port isp0
            port isp1
            lun 0 fibre-channel
    }

On the mac side I see the vendor and product (really only added
them there for debugging), but I certainly do not see the serial
number.

    emerald:aram$ sudo mpioutil list
        Alias            Vendor           Product          LUN Identifier
                   Algorithm
      1 No Alias         MGK              FC Disk
                   Least Bytes
      2 No Alias         MGK              FC Disk
                   Least Bytes
    emerald:aram$

The "LUN Identifier" is blank. Comparing the output with output
from people who have this working with Solaris+mac (instead of
FreeBSD) I see that there should only be one entry there if multipath
is working, and the "LUN Identifier" should not be blank.

The only other debugging information the mac provides is the "system
report", which in my case says:

    Capacity:   912,68 GB (912.680.550.400 bytes)
    LUN Address:    00:00:00:00:00:00:00:00
    Manufacturer:   MGK
    Model:  FC Disk
    Revision:   1
    ALUA Support:   Implicit
    Logical Unit Access State:  Active Optimized
    Load Balancing Algorithm:   Least Bytes
    Removable Media:    No
    Detachable Drive:   No
    BSD Name:   disk2
    Partition Map Type: Unknown

A good one should be:

    Capacity: 8 TB (7,999,447,891,968 bytes)
    LUN Address:  00:00:00:00:00:00:00:00
    Logical Unit Unique ID:   0x600A0B80002682A8000067D95897A544
    Manufacturer: SUN
    Model:    CSM200_R
    Revision: 760
    ALUA Support: Unsupported
    Logical Unit Access State:    Active Optimized
    Load Balancing Algorithm: Least Bytes
    Removable Media:  Yes
    Detachable Drive: No
    BSD Name: disk6
    Partition Map Type:   GPT (GUID Partition Table)
    S.M.A.R.T. status:    Not Supported

Notice that I am missing the "Logical Unit Unique ID" while the
working configuration displays it.

Am I doing something wrong here? Why doesn't the serial number
and/or the device-id I specified in ctl.conf propagate to the
initiator?  Unfortunately the mac really can't provide any more
debugging information than that. Unless I use DTrace, but without
source it's pretty difficult.

On the FreeBSD side I see this type of output:

    ctlfe_onoffline: isp0 current WWNN 0x2000001b321cd5b6
    ctlfe_onoffline: isp0 current WWPN 0x2100001b321cd5b6
    isp0: Setting Channel 0 wwns to 0x2100001b321cd5b6 0x2100001b321cd5b6
    ctlfe_onoffline: isp0 new WWNN 0x2100001b321cd5b6
    ctlfe_onoffline: isp0 new WWPN 0x2100001b321cd5b6
    ctlfe_onoffline: isp0 (path id 6) target role enable succeeded
    ctlfe_onoffline: isp1 current WWNN 0x2000001b323cd5b6
    ctlfe_onoffline: isp1 current WWPN 0x2101001b323cd5b6
    isp1: Setting Channel 0 wwns to 0x2100001b321cd5b6 0x2101001b323cd5b6
    ctlfe_onoffline: isp1 new WWNN 0x2100001b321cd5b6
    ctlfe_onoffline: isp1 new WWPN 0x2101001b323cd5b6
    ctlfe_onoffline: isp1 (path id 7) target role enable succeeded
    ctlfeasync: WWPN 0x2802000155535323 port 0x000001 path 6 target 0
arrived
    ctlfeasync: WWPN 0x2801000155535323 port 0x000001 path 7 target 0
arrived

But, as I have never done this before, I don't know if that looks
correct or not. I don't know if "path 6" and "path 7" there should
be the same or not.

I can enable more debugging information on the FreeBSD side, but
at the moment I don't know what to look for.

Just for reference, I also tried this config file, and it made no
difference, I get the same behavior with it:

    lun fibre-channel {
            backend block
            device-id "FC Disk AAAA0000ZZZ01"
            serial "AAAA0000ZZZ01"
            option vendor "MGK"
            option product "FC Disk"
            path /dev/zvol/tank1/export/fc
    }

    target naa.2100001b321cd5b6 {
            auth-group no-authentication
            port isp0
            lun 0 fibre-channel
    }

    target naa.2101001b323cd5b6 {
            auth-group no-authentication
            port isp1
            lun 0 fibre-channel
    }

Thoughts?

Thanks,

-- 
Aram Hăvărneanu