QLogic 24xx FC HBAs not working fine with CAM Target Layer

Wed Jul 23 17:39:31 UTC 2014

On Wed, Jul 23, 2014 at 10:19 PM, Kenneth D. Merry <ken at freebsd.org> wrote:

> On Wed, Jul 23, 2014 at 12:33:16 +0530, bharat singh wrote:
> > On Wed, Jul 23, 2014 at 9:52 AM, Kenneth D. Merry <ken at freebsd.org>
> wrote:
> >
> > > On Tue, Jul 22, 2014 at 13:06:53 +0530, bharat singh wrote:
> > > > Hello,
> > > >
> > > > I am trying to use a new firmware on a single port FC HBA with
> freebsd9
> > > > stable, but facing the firmware issue.
> > > >
> > > > Card came with firmware version 5.5.0
> > > > (isp0: Board Type 2422, Chip Revision 0x3, resident F/W Revision
> 5.5.0)
> > > >
> > > > I upgraded the firmware to 5.6.5, and disabled firmware flag by
> setting
> > > > "fwload_disable=YES" flag in /boot/device.hints to retain the newer
> > > > firmware.
> > > > On subsequent reboots the firmware fails to initialize and throws
> > > > isp0: Mailbox Command 'ABOUT FIRMWARE' failed (TIMEOUT)
> > >
> > > In general, you want to load the ispfw module or compile it into your
> > > kernel.  The driver was tested with the firmware included in the
> module,
> > > and may not work as well (or at all) with a different version.
> > >
> > > If you know what changed between the firmware version compiled into the
> > > ispfw module and the one you want to run, you can make any necessary
> driver
> > > modifications.
> > >
> > > > [root at Ctrl4 ~]# ctladm port -l
> > > > Port Online Type     Name         pp vp WWNN               WWPN
> > > >    Speed  Vendor
> > > > 0    YES    IOCTL    CTL ioctl    0  0  0                  0
> > > >     0
> > > > 1    YES    INTERNAL ctl2cam      0  0  0x5000000edac0c700
> > > > 0x5000000edac0c702  0
> > > > 2    YES    INTERNAL CTL internal 0  0  0                  0
> > > >     0
> > > >
> > > > * It's not showing FC port
> > >
> > > You need to compile your kernel with:
> > >
> > > options         ISP_TARGET_MODE=1
> > >
> > > Otherwise you won't get target mode functionality in the isp(4) driver.
> > >
> > Yes, ISP is enabled for target functionality.
>
> In order for CTL to recognize that an HBA is target-capable, the driver has
> to set the PIT_PROCESSOR bit (at least) in the target_sprt field in the
> XPT_PATH_INQ CCB.
>
> If you don't see the driver listed in the CTL port display, that means that
> the driver likely isn't reporting itself to be target-capable.
>
> See the code in isp_freebsd.c that fills in the XPT_PATH_INQ CCB.
>
> > > > 1448 Jul 22 11:05:37 Ctrl4 kernel: isp0: IN mbox 0 = 0x0008
> > > > 1450 Jul 22 11:05:37 Ctrl4 kernel: isp0: RISC2HOST ISR 0x40000101
> > > > 1451 Jul 22 11:05:37 Ctrl4 last message repeated 999 times
> > > > 1454 Jul 22 11:05:37 Ctrl4 kernel: isp0: Polled Mailbox Command (0x8)
> > > > Timeout (100000us) (started @ isp_reset:1045)
> > > > 1456 Jul 22 11:05:37 Ctrl4 kernel: isp0: Mailbox Command 'ABOUT
> FIRMWARE'
> > > > failed (TIMEOUT)
> > > > 1458 Jul 22 11:05:37 Ctrl4 kernel: isp0: RISC2HOST ISR 0x40000101
> > > > 1460 Jul 22 11:05:37 Ctrl4 kernel: Trying to mount root from
> > > > ufs:/dev/label/root_0 [rw]...
> > > > 1462 Jul 22 11:05:37 Ctrl4 kernel: device_attach: isp0 attach
> returned 6
> > > > 1464 Jul 22 11:05:37 Ctrl4 kernel: isp0: <Qlogic ISP 2432 PCI FC-AL
> > > > Adapter> port 0x3000-0x30ff mem 0xc1040000-0xc1043fff irq 16 at
> device
> > > 0.0
> > > > on pci     3
> > > > 1465 Jul 22 11:05:37 Ctrl4 kernel: isp0: loaded firmware
> isp_2400_multi
> > > >
> > > > from dmesg:
> > > > isp0: RISC2HOST ISR 0x40000101
> > > > isp0: Polled Mailbox Command (0x8) Timeout (100000us) (started @
> > > > isp_reset:1045)
> > > > isp0: Mailbox Command 'ABOUT FIRMWARE' failed (TIMEOUT)
> > > > isp0: RISC2HOST ISR 0x40000101
> > > > device_attach: isp0 attach returned 6
> > > > isp0: <Qlogic ISP 2432 PCI FC-AL Adapter> port 0x3000-0x30ff mem
> > > > 0xc1040000-0xc1043fff irq 16 at device 0.0 on pci3
> > > > isp0: loaded firmware isp_2400_multi
> > > > isp0: set PCI latency to 64
> > > > isp0: request/result area @ 0xf700000/0x38000
> > > > isp0: scdma @ 0xf6b9000/0x1000
> > > > isp0: IN mbox 0 = 0x0000
> > > > isp0: RISC2HOST ISR 0x0
> > > > isp0: RISC2HOST ISR 0x40008001
> > > > isp0: OUT mbox 0 = 0x4000
> > > > isp0: IN mbox 0 = 0x0006
> > > > isp0: IN mbox 1 = 0xdead
> > > >
> > > > The workaround is to enable the firmware flash by setting
> > > > "fwload_disable=NO", but that doesn't serves my purpose.
> > > >
> > > > Is there a way to avoid the issue without a firmware flash on every
> > > reboot ?
> > > > Moreover i have seen the issue on older firmware 5.5.0 also, as soon
> I
> > > > disable the firmware flash and reboot.
> > >
> > > If I recall correctly, the firmware that is loaded on boot is loaded
> into
> > > RAM on the card, not flashed.  I don't think it affects what is
> actually
> > > flashed onto the board.
> >
> > When I disable the firmware download by setting "fwload_disable=YES" and
> > reboot, still most of the times it works. So i think its the eprom
> getting
> > overwritten on every reboot.
>
> Take a look at isp_reset() in isp.c.  It downloads firmware into RAM on the
> card, and then tells the card to execute it.  It isn't re-flashing the
> card.
>
> If that were the case, you would have to use a tool to flash new firmware
> to get back to the newer version of firmware.  Instead, you just tell the
> driver not to download firmware and it uses the firmware that's flashed
> into the card.
>
> Yes isp_reset() skips firmware download if firmware is not there or its
overridden by the flag. In my setup I haven't flashed the firmware. After I
load the newer firmware 5.6.5 and suppress further downloads, the new
firmware exists even after reboots. Does it means that my HBA card came
with 5.6.5 or the loaded firmware get persistent somewhere.

> >
> > So that really is what you want to do.
> > >
> >
> > I disabled ISP, compiled the kernel with Qlogic driver and loaded its
> > firmware. Since the Qlogic driver is not compatible with our CTL module,
> I
> > need to go back to the old ISP driver. So I reverted my kernel and
> > suppressed further firmware downloads by "fwload_disable=YES" and tried
> > out. Having new firmware with old ISP driver is creating issues (mailbox
> > timeouts)
>
> Which Qlogic driver are you using?
>

I am trying to use a Qlogic version 5.6.5, but it doesn't integrate with
the CTL of freebsd. It  has all the functionality to manage the Qlogic card
“messages:May 26 17:52:33 es175 kernel: isp0: Board Type 2532, Chip
Revision 0x2, resident F/W Revision 5.6.5”

> If it isn't compatible with CTL, then it will likely take some work to add
> the ability to interface with CTL.
>

> > >
> > > Also, just FYI, in past testing with 4Gb Qlogic boards I was limited to
> > > 256MB/sec in each direction when I tried them out.  I haven't seen the
> same
> > > limitation with the 8Gb boards.
> > >
> > > It has been a few years since I tried out target mode with a 4Gb
> board, but
> > > keep an eye out for that once you get it running.
> > >
> > > Ken
> > > --
> > > Kenneth Merry
> > > ken at FreeBSD.ORG
> > >
> >
> >
> >
> > --
> > Bharat Singh
> > Member Technical Staff, NetApp
>
> Ken
> --
> Kenneth Merry
> ken at FreeBSD.ORG
>

-- 
Bharat Singh
Member Technical Staff, NetApp