From nobody Wed Aug 02 01:10:26 2023 X-Original-To: questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RFv7j0M2sz4q4SK for ; Wed, 2 Aug 2023 01:10:39 +0000 (UTC) (envelope-from rogan.mcgovern@gmail.com) Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com [IPv6:2a00:1450:4864:20::133]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RFv6k38Fgz3rLv for ; Wed, 2 Aug 2023 01:10:38 +0000 (UTC) (envelope-from rogan.mcgovern@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20221208 header.b=Hl6aHLMD; spf=pass (mx1.freebsd.org: domain of rogan.mcgovern@gmail.com designates 2a00:1450:4864:20::133 as permitted sender) smtp.mailfrom=rogan.mcgovern@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-lf1-x133.google.com with SMTP id 2adb3069b0e04-4fe4762173bso1897125e87.3 for ; Tue, 01 Aug 2023 18:10:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1690938635; x=1691543435; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=slDtUr/HzFWietj9aYcmUbiy25e1qkxpEvnBQWn//Bs=; b=Hl6aHLMD6NY3s4oXSnrD5yFcWqjDrsOBi5p96hyCMPaerqbscV2oLKRJKIztzOByme avohqzCOrnMWX0ol++96k5IBAUFB6mDLqidUlRXp0jGtDYNLSTiirCe2FXGAhMC74ANA 1rz0hw9lHw7lnTdY4TSsU7iN2QKKHH3nKiKPPsdMa+w1FZD1ElKiKkn5xk38qpl8PZBC 4x5pz3xkU+cEmoUH0xoabxrYD6yb7jIm6Dn7AFchbb4wy7T3J0G6kRZ0ZOSspS9AXjeP mr/EzGqlB7cgsmrxYUi2TyuuqNRNYhJE3WzpIKMWkW3UxLbCWfs3UdUxTMHkcWh1J0si zSpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690938635; x=1691543435; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=slDtUr/HzFWietj9aYcmUbiy25e1qkxpEvnBQWn//Bs=; b=UuovGi7lUbGA6XHhHCkTmlkeRGBtbUNi2QRcWRoTRNcxTrOk4uoOqWB++qz+69DHZt tce8BjnNTA4eAXnTmAlIfzLaiERcXOoopStIziAslAaLArDhfeqxzQR6W7HkrKF3x5Y2 u+RCiwmN5g/vVxwIkvwKnVfrA0RcZc4gv0j5vZdbr+9soK/AUDyncXSl4NqBXiX8PJjS ftq14GTSqaMIH/Ms2tzAvtiSNfA4cP6/jXM5/jAwFPRxArBTGf5q6G9XSSdmR0x5tccL UV1zKEql3RB3zMFLftlW+OKvdlohMi4VlvDHo5EW2LX/ykMVl9JgtFNTz+C3R6gGlVB2 A70g== X-Gm-Message-State: ABy/qLavU3SJAmWOfRE9MKDk+ohXfsMdXy1JCwyo11jsl83rKZx1W1OP sWXRcq6ImhH9UFwzgRG4t9x0UIEp2NRN2fp6ZGA= X-Google-Smtp-Source: APBJJlFPJPpBcAFgDiiuAHNJDpYUYu4sBlwBPQv5TKBCW6bfyAn11ZQOikHicM6RZo2Rjyomnkhyu+h1i12XEg/vuf4= X-Received: by 2002:a05:6512:3c84:b0:4fd:d6ff:c244 with SMTP id h4-20020a0565123c8400b004fdd6ffc244mr4387998lfv.62.1690938634918; Tue, 01 Aug 2023 18:10:34 -0700 (PDT) List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org MIME-Version: 1.0 References: <5cf2f487-a5d2-55b7-49df-51690ac22412@aetern.org> <6on92p68-rpq2-594p-8q5s-979ns8p12653@sez2.ghz.qr> In-Reply-To: <6on92p68-rpq2-594p-8q5s-979ns8p12653@sez2.ghz.qr> From: Brian McGovern Date: Tue, 1 Aug 2023 21:10:26 -0400 Message-ID: Subject: Re: isp support for QLogic 2722/firmware? To: Joerg Pulz Cc: questions@freebsd.org Content-Type: multipart/alternative; boundary="000000000000abde550601e654d4" X-Spamd-Result: default: False [-2.72 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-0.999]; NEURAL_HAM_MEDIUM(-0.88)[-0.884]; NEURAL_HAM_SHORT(-0.84)[-0.836]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20221208]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[questions@freebsd.org]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::133:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; TAGGED_FROM(0.00)[]; MLMMJ_DEST(0.00)[questions@freebsd.org]; DKIM_TRACE(0.00)[gmail.com:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; RCPT_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4RFv6k38Fgz3rLv X-Spamd-Bar: -- --000000000000abde550601e654d4 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Joerg, Sorry it took me a few days to get back to you and the list. Thank you for answering the questions I asked. It required a firmware upgrade that had previously caused problems with VMWare's ESXi. The fact that someone was using the adapters, despite not a lot mentioned over the last few years during my Google search, meant I wasn't wasting my time chasing a problem that had no solution. Mentioning the card should be using the onboard firmware pointed me in the right direction. Everything appears to be working now. The only open issue I have is making sure that the multi-path is working when I get the machine moved to the production network. -Brian On Thu, Jul 27, 2023 at 5:40=E2=80=AFAM Joerg Pulz = wrote: > On Wed, 26 Jul 2023, Brian McGovern wrote: > > > >> I'm curious if anyone has this working, and if so, any chance I can > >>> obtain the firmware or process you are using? I'm hoping its somethin= g > >>> like flashing an image to the card with some other OS first. TIA > > > >> Support for this (and some other newer HBAs) was added recently to > >> 14-CURRENT, see latest commits in > >> https://cgit.freebsd.org/src/log/sys/dev/isp > > . > > > > Its getting late locally, so take this with a grain of salt... I'm pret= ty > > sure I grabbed the main branch from git, and effectively did a MFC > (working > > with 13.2-RELEASE). Looking at the diffs you pointed me to vs. whats on > the > > disk, it looks like I have the latest bits, but I'll check it again > > tomorrow to be sure. Its possible I'm missing something. > > > > The problem is consistent. When the adapters try to come up, and I'm > using > > them as kernel modules right now just so I can cycle through testing > > faster, I continue to get > > > > "isp_2700 could not load firmware image, error 2" > > > > if I don't hack up the ispfw driver/module to have something that meets > the > > name requirements for "isp_2700" firmware. > > > > and if I try to create a firmware module it can install with the > > bk014519.bin firmware file, it'll happily find something for firmware, > but > > then throws a: > > isp0: Mailbox Command (0xb) Timeout (100000us) (isp_reset:373) > > isp0: Mailbox Command 'LOAD RISC RAM' failed (TIMEOUT) > > isp0: F/W download failed > > isp0: isp_reinit: cannot reset card > > device_attach: isp0 attach returned 6 > > > > So, what I think is missing is either a.) finding a good firmware image > > that I can load via the driver, or b.) figuring out how to flash firmwa= re > > to the adapter via some other means, and figure out how to skip the loa= d > > step when the driver starts. Hence why I'm hoping to find someone who h= as > > the card working, so I can clone their process. > > > > But, if anyone has other ideas, I'm all ears. > > > > -Brian > > > > On Wed, Jul 26, 2023 at 8:55?PM Yuri wrote: > > > >> Brian McGovern wrote: > >>> Quick question about the isp and card support. I have an HPE branded > >>> Qlogic ISP 2722 that I'm trying to bring online. Following the variou= s > >>> tutorials I loaded ispfw, then the isp driver. I got an error message > >>> "isp_2700 could not load firmware image, error 2". The boards don't > come > >>> online. > >>> > >>> Looking at the man page, it appears supported on the Qlogic > >>> 2740/2742/2764 (aka 2722/2714) line, so I tried to chase down some > >>> firmware and duplicate the isp_2400 and isp_2500 functionality with t= he > >>> .bin file. Now, it tries to load but I get a firmware loading failure= . > >>> I'm assuming that either current versions and .bin files are > >>> incompatible with the other loading processes, or I just screwed > >>> something up in all the translation (I found a bk014519.bin file). > >>> > >>> I'm curious if anyone has this working, and if so, any chance I can > >>> obtain the firmware or process you are using? I'm hoping its somethin= g > >>> like flashing an image to the card with some other OS first. TIA > >> > >> Support for this (and some other newer HBAs) was added recently to > >> 14-CURRENT, see latest commits in > >> https://cgit.freebsd.org/src/log/sys/dev/isp. > > Those HBAs are supported since 2018-02-28. > So support for them is in 13.2 - I have systems running with those: > > # uname -r > 13.2-RELEASE > > # dmesg -a |grep isp > isp0: mem > 0x93302000-0x93302fff,0x93300000-0x93301fff,0x93200000-0x932fffff at devi= ce > 0.0 on pci1 > > # sysctl dev.isp.0 > dev.isp.0.%pnpinfo: vendor=3D0x1077 device=3D0x2261 subvendor=3D0x1077 > subdevice=3D0x029b class=3D0x0c0400 > dev.isp.0.%driver: isp > dev.isp.0.%desc: Qlogic ISP 2722 PCI FC Adapter > > Recent changes in CURRENT are for the next gen HABs (28XX) and FLT/NVRAM > handling, so no need to backport anything. > > There is no firmware for those HBAs included in ispfw(4) and there is no > need for it. > You should not try to simply add firmware to ispfw(4) without knowing how > the driver has to load it into the HBA. > > The isp(4) driver checks for firmware availability by itself. For your > HBA the firmware is loaded by isp(4) from the HBAs flash. > If this is not working than the firmware on your HBAs flash may be broken > and should be reflashed. > > Without further information it is hard to guess what's happening. > You should: > - revert everything to a clean 13.2-RELEASE > - provide verbose pciconf(8) output for your HBA > - set isp(4) debug level using kenv(8) > # kenv hint.isp.0.debug=3D0x3f > - load isp(4) > # kldload isp > - provide dmesg(8) output for isp(4) load/attach > > Joerg > > -- > The beginning is the most important part of the work. > -Plato --000000000000abde550601e654d4 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Joerg,
=C2=A0 Sorry it took me a few days t= o get back to you and the list. Thank you for answering the questions I ask= ed. It required a firmware upgrade that had previously caused problems with= VMWare's ESXi.

=C2=A0 The fact that someone w= as using the adapters, despite not a lot mentioned over the last few years = during my Google search, meant I wasn't wasting my time chasing a probl= em that had no solution. Mentioning the card should be using the onboard fi= rmware pointed me in the right direction.

=C2= =A0 Everything appears to be working now. The only open issue I have is mak= ing sure that the multi-path is working when I get the machine moved to the= production network.

=C2=A0=C2=A0=C2=A0=C2=A0 -Bri= an

On Thu, Jul 27, 2023 at 5:40=E2=80=AFAM Joerg Pulz <Joerg.Pulz@frm2.tum.de> wrote:
On Wed, 26 Jul 202= 3, Brian McGovern wrote:

> >> I'm curious if anyone has this working, and if so, any ch= ance I can
>>> obtain the firmware or process you are using? I'm hoping i= ts something
>>> like flashing an image to the card with some other OS first. T= IA
>
>> Support for this (and some other newer HBAs) was added recently to=
>> 14-CURRENT, see latest commits in
>> https://cgit.freebsd.org/src/log/sys/dev/isp
> <https://cgit.freebsd.org/src/log/sys/dev/isp>.
>
> Its getting late locally, so take this with a grain of salt... I'm= pretty
> sure I grabbed the main branch from git, and effectively did a MFC (wo= rking
> with 13.2-RELEASE). Looking at the diffs you pointed me to vs. whats o= n the
> disk, it looks like I have the latest bits, but I'll check it agai= n
> tomorrow to be sure. Its possible I'm missing something.
>
> The problem is consistent. When the adapters try to come up, and I'= ;m using
> them as kernel modules right now just so I can cycle through testing > faster, I continue to get
>
> "isp_2700 could not load firmware image, error 2"
>
> if I don't hack up the ispfw driver/module to have something that = meets the
> name requirements for "isp_2700" firmware.
>
> and if I try to create a firmware module it can install with the
> bk014519.bin firmware file, it'll happily find something for firmw= are, but
> then throws a:
> isp0: Mailbox Command (0xb) Timeout (100000us) (isp_reset:373)
> isp0: Mailbox Command 'LOAD RISC RAM' failed (TIMEOUT)
> isp0: F/W download failed
> isp0: isp_reinit: cannot reset card
> device_attach: isp0 attach returned 6
>
> So, what I think is missing is either a.) finding a good firmware imag= e
> that I can load via the driver, or b.) figuring out how to flash firmw= are
> to the adapter via some other means, and figure out how to skip the lo= ad
> step when the driver starts. Hence why I'm hoping to find someone = who has
> the card working, so I can clone their process.
>
> But, if anyone has other ideas, I'm all ears.
>
>=C2=A0 =C2=A0 =C2=A0-Brian
>
> On Wed, Jul 26, 2023 at 8:55?PM Yuri <yuri@aetern.org> wrote:
>
>> Brian McGovern wrote:
>>> Quick question about the isp and card support. I have an HPE b= randed
>>> Qlogic ISP 2722 that I'm trying to bring online. Following= the various
>>> tutorials I loaded ispfw, then the isp driver. I got an error = message
>>> "isp_2700 could not load firmware image, error 2". T= he boards don't come
>>> online.
>>>
>>> Looking at the man page, it appears supported on the Qlogic >>> 2740/2742/2764 (aka 2722/2714) line, so I tried to chase down = some
>>> firmware and duplicate the isp_2400 and isp_2500 functionality= with the
>>> .bin file. Now, it tries to load but I get a firmware loading = failure.
>>> I'm assuming that either current versions and .bin files a= re
>>> incompatible with the other loading processes, or I just screw= ed
>>> something up in all the translation (I found a bk014519.bin fi= le).
>>>
>>> I'm curious if anyone has this working, and if so, any cha= nce I can
>>> obtain the firmware or process you are using? I'm hoping i= ts something
>>> like flashing an image to the card with some other OS first. T= IA
>>
>> Support for this (and some other newer HBAs) was added recently to=
>> 14-CURRENT, see latest commits in
>> https://cgit.freebsd.org/src/log/sys/dev/isp.

Those HBAs are supported since 2018-02-28.
So support for them is in 13.2 - I have systems running with those:

# uname -r
13.2-RELEASE

# dmesg -a |grep isp
isp0: <Qlogic ISP 2722 PCI FC Adapter> mem 0x93302000-0x93302fff,0x93= 300000-0x93301fff,0x93200000-0x932fffff at device 0.0 on pci1

# sysctl dev.isp.0
dev.isp.0.%pnpinfo: vendor=3D0x1077 device=3D0x2261 subvendor=3D0x1077 subd= evice=3D0x029b class=3D0x0c0400
dev.isp.0.%driver: isp
dev.isp.0.%desc: Qlogic ISP 2722 PCI FC Adapter

Recent changes in CURRENT are for the next gen HABs (28XX) and FLT/NVRAM handling, so no need to backport anything.

There is no firmware for those HBAs included in ispfw(4) and there is no need for it.
You should not try to simply add firmware to ispfw(4) without knowing how <= br> the driver has to load it into the HBA.

The isp(4) driver checks for firmware availability by itself. For your
HBA the firmware is loaded by isp(4) from the HBAs flash.
If this is not working than the firmware on your HBAs flash may be broken <= br> and should be reflashed.

Without further information it is hard to guess what's happening.
You should:
- revert everything to a clean 13.2-RELEASE
- provide verbose pciconf(8) output for your HBA
- set isp(4) debug level using kenv(8)
=C2=A0 =C2=A0# kenv hint.isp.0.debug=3D0x3f
- load isp(4)
=C2=A0 =C2=A0# kldload isp
- provide dmesg(8) output for isp(4) load/attach

Joerg

--
The beginning is the most important part of the work.
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 -Plato
--000000000000abde550601e654d4--