From nobody Fri Jun 02 05:28:07 2023 X-Original-To: wireless@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QXWkj497Bz4YWK8 for ; Fri, 2 Jun 2023 05:28:45 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from webmail5.jnielsen.net (webmail5.jnielsen.net [IPv6:2607:f170:34:11::b0]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.freebsdsolutions.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QXWkb60Jcz47PJ; Fri, 2 Jun 2023 05:28:39 +0000 (UTC) (envelope-from lists@jnielsen.net) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of lists@jnielsen.net designates 2607:f170:34:11::b0 as permitted sender) smtp.mailfrom=lists@jnielsen.net; dmarc=none Received: from smtpclient.apple ([50.207.241.62]) (authenticated bits=0) by webmail5.jnielsen.net (8.17.1/8.17.1) with ESMTPSA id 3525SM3T073953 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 1 Jun 2023 23:28:30 -0600 (MDT) (envelope-from lists@jnielsen.net) X-Authentication-Warning: webmail5.jnielsen.net: Host [50.207.241.62] claimed to be smtpclient.apple From: John Nielsen Message-Id: <8CEDC118-5FCA-434D-B7F7-B0C4A7AC3ACC@jnielsen.net> Content-Type: multipart/alternative; boundary="Apple-Mail=_0601CF93-4FB8-43DE-9BAD-B9075B1001E7" List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-wireless List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-wireless@freebsd.org X-BeenThere: freebsd-wireless@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.600.7\)) Subject: Re: Help me grok the ath(4) device attach code Date: Thu, 1 Jun 2023 23:28:07 -0600 In-Reply-To: <5264D290-EB82-4D24-A812-85E3E6B5C88E@jnielsen.net> Cc: "Bjoern A. Zeeb" , "wireless@freebsd.org" To: Adrian Chadd References: <49AEA1CB-FA85-432F-89D7-8C49B5F3A344@jnielsen.net> <85BF3AB2-2EBF-4398-A507-ABA35505A56C@jnielsen.net> <5264D290-EB82-4D24-A812-85E3E6B5C88E@jnielsen.net> X-Mailer: Apple Mail (2.3731.600.7) X-Spamd-Result: default: False [-2.80 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+a:mailers.freebsdsolutions.net]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[jnielsen.net]; MLMMJ_DEST(0.00)[wireless@freebsd.org]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:6364, ipnet:2607:f170:30::/44, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:~]; BLOCKLISTDE_FAIL(0.00)[2607:f170:34:11::b0:server fail,50.207.241.62:query timed out]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCPT_COUNT_THREE(0.00)[3]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_SOME(0.00)[]; HAS_XAW(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4QXWkb60Jcz47PJ X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_0601CF93-4FB8-43DE-9BAD-B9075B1001E7 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On May 31, 2023, at 1:52 PM, John Nielsen wrote: >=20 >> On May 30, 2023, at 11:17 PM, Adrian Chadd = wrote: >>=20 >> On Tue, 30 May 2023 at 22:12, John Nielsen > wrote: >>>> On May 30, 2023, at 10:56 PM, Adrian Chadd > wrote: >>>>=20 >>>> On Tue, 30 May 2023 at 20:56, John Nielsen > wrote: >>>>> > On May 30, 2023, at 8:02 PM, Adrian Chadd > wrote: >>>>> >=20 >>>>> > Err, if it's coming up w/ that MAC then it's not finding and = attaching right to the OTP/EEPROM calibration information. That's the = big red flag that it in general won't work correctly. >>>>> >=20 >>>>> > Can you provide the rest of the ath_hal messages? I'd like to = see what it's saying during boot around it checking the EEPROM/OTP = contents. It's possible there's some work around required for this NIC. >>>>>=20 >>>>> He speaks! Thanks for taking the time. I just realized that = ath_hal_printf doesn=E2=80=99t prepend =E2=80=9Cath%d=E2=80=9D so I=E2=80=99= ve been missing those messages when grep-ing. Here=E2=80=99s the whole = snippet: >>>>>=20 >>>>> ath0: mem 0xf7a00000-0xf7a7ffff at device = 0.0 on pci4 >>>>> ar9300_flash_map: unimplemented for now >>>>> Restoring Cal data from DRAM >>>>> Restoring Cal data from EEPROM >>>>> Restoring Cal data from Flash >>>>> Restoring Cal data from Flash >>>>> Restoring Cal data from OTP >>>>> ar9300_eeprom_restore_internal[4338] No vaid CAL, calling default = template >>>>> ar9300_hw_attach: ar9300_eeprom_attach returned 0 >>>>=20 >>>> Yeah, this bit right here is the problem. It's not finding a valid = calibration. >>>=20 >>>> oh err, is there a wifi enable/disable switch or something? maybe = it's asserted and somehow it's mucking up the NIC? >>>=20 >>> There is a physical switch and it=E2=80=99s in the =E2=80=9Cenable=E2=80= =9D position. >>>=20 >>>> I wonder what ath9k is doing here? Is there some weird pci based = workaround/flag for the given NIC PCI id? >>>=20 >>> That was the first breadcrumb BZ threw me but I can=E2=80=99t find = anything. There are some .driver_data hints for adjacent subdevice IDs = but none for this one (Dell 0x020d) in either FreeBSD or Linux that I = could find. >>>=20 >>> [snip] >>> There is this commit in ath9k which mentions an alternative EEPROM = address, but I=E2=80=99m not sure if that=E2=80=99s relevant. =46rom = what I can tell the probe should succeed at the normal base_address = 0x3ff instead of needing to try the =E2=80=9C4k=E2=80=9D one 0xfff. >>> = https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/commit/drive= rs/net/wireless/ath/ath9k?id=3D528782ecf59f7bab2f1368628a479f49be59b512 >>=20 >> Yeah i'd try that. It'd be nice if I knew that the NIC used OTP or = EEPROM though. >>=20 >> There's known issues with all the Atheros chips (sigh) with how the = EEPROM and PCIe bus reset .. interact. >> (If the bus reset is too short then the EEPROM state machine gets = stuck and nothing gets read.) It makes debugging this hard because the = NIC itself will work in another device fine, because it's the BIOS/ACPI = code. :( >=20 > The 4k EEPROM read didn=E2=80=99t work. But I did notice that where it = says it=E2=80=99s restoring Cal data from OTP it actually just does an = EEPROM read again. Shouldn=E2=80=99t this line be a call to = ar9300_otp_read() instead of ar9300_eeprom_restore_internal_address()? >=20 > = https://cgit.freebsd.org/src/tree/sys/contrib/dev/ath/ath_hal/ar9300/ar930= 0_eeprom.c#n4292 >=20 > Otherwise it doesn=E2=80=99t use the 0x14000 (0x30000 for some cards) = OTP offset as a starting point. I=E2=80=99d love to know if I=E2=80=99m up in the night about my theory = above about the OTP call not actually happening. In the meantime I=E2=80=99= ll carry on either trying to find/build a Linux kernel with = CONFIG_ATH_DEBUG enabled that I can boot on the laptop (to see which = EEPROM retrieval/check actually succeeds) and/or try to reimplement bits = from the ath9k ar9300_eeprom_restore_internal() to match its behavior on = FreeBSD. ( = https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/tree/drivers= /net/wireless/ath/ath9k/ar9003_eeprom.c#n3266 ) Thanks, JN --Apple-Mail=_0601CF93-4FB8-43DE-9BAD-B9075B1001E7 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
On May 31, 2023, = at 1:52 PM, John Nielsen <lists@jnielsen.net> wrote:

On May 30, 2023, at 11:17 PM, Adrian Chadd = <adrian@freebsd.org> wrote:

On Tue, 30 = May 2023 at 22:12, John Nielsen <lists@jnielsen.net> = wrote:
On May 30, 2023, at 10:56 = PM, Adrian Chadd <adrian@freebsd.org> wrote:

On Tue, 30 = May 2023 at 20:56, John Nielsen <lists@jnielsen.net> wrote:
> On May 30, 2023, at 8:02 = PM, Adrian Chadd <adrian@freebsd.org> = wrote:
> 
> Err, if it's coming up w/ that = MAC then it's not finding and attaching right to the OTP/EEPROM = calibration information. That's the big red flag that it in general = won't work correctly.
> 
> Can you provide = the rest of the ath_hal messages? I'd like to see what it's saying = during boot around it checking the EEPROM/OTP contents. It's possible = there's some work around required for this NIC.

He speaks! Thanks = for taking the time. I just realized that ath_hal_printf doesn=E2=80=99t = prepend =E2=80=9Cath%d=E2=80=9D so I=E2=80=99ve been missing those = messages when grep-ing. Here=E2=80=99s the whole snippet:

ath0: = <Atheros AR946x/AR948x> mem 0xf7a00000-0xf7a7ffff at device 0.0 on = pci4
ar9300_flash_map: unimplemented for now
Restoring Cal data = from DRAM
Restoring Cal data from EEPROM
Restoring Cal data from = Flash
Restoring Cal data from Flash
Restoring Cal data from = OTP
ar9300_eeprom_restore_internal[4338] No vaid CAL, calling default = template
ar9300_hw_attach: ar9300_eeprom_attach returned = 0

Yeah, this bit right here is the = problem. It's not finding a valid = calibration.

oh err, is there a wifi enable/disable switch or = something? maybe it's asserted and somehow it's mucking up the = NIC?

There is a physical switch and = it=E2=80=99s in the =E2=80=9Cenable=E2=80=9D = position.

 I wonder what ath9k is doing here? Is = there some weird pci based workaround/flag for the given NIC PCI = id?

That was the first = breadcrumb BZ threw me but I can=E2=80=99t find anything. There are some = .driver_data hints for adjacent subdevice IDs but none for this one = (Dell 0x020d) in either FreeBSD or Linux that I could = find.

[snip]
There is this commit in ath9k = which mentions an alternative EEPROM address, but I=E2=80=99m not sure = if that=E2=80=99s relevant. =46rom what I can tell the probe should = succeed at the normal base_address 0x3ff instead of needing to try the = =E2=80=9C4k=E2=80=9D one 0xfff.

Yeah i'd = try that. It'd be nice if I knew that the NIC used OTP or EEPROM = though.

There's known issues with all the = Atheros chips (sigh) with how the EEPROM and PCIe bus reset .. = interact.
(If the bus reset is too short then the EEPROM state = machine gets stuck and nothing gets read.) It makes debugging this hard = because the NIC itself will work in another device fine, because it's = the BIOS/ACPI code. :(

The 4k EEPROM = read didn=E2=80=99t work. But I did notice that where it says it=E2=80=99s= restoring Cal data from OTP it actually just does an EEPROM read again. = Shouldn=E2=80=99t this line be a call to ar9300_otp_read() instead of = ar9300_eeprom_restore_internal_address()?


Otherwise it = doesn=E2=80=99t use the 0x14000 (0x30000 for some cards) OTP offset as a = starting point.

I=E2=80=99d love = to know if I=E2=80=99m up in the night about my theory above about the = OTP call not actually happening. In the meantime I=E2=80=99ll carry on = either trying to find/build a Linux kernel with CONFIG_ATH_DEBUG enabled = that I can boot on the laptop (to see which EEPROM retrieval/check = actually succeeds) and/or try to reimplement bits from the ath9k = ar9300_eeprom_restore_internal() to match its behavior on = FreeBSD.
( = https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/tree/drivers= /net/wireless/ath/ath9k/ar9003_eeprom.c#n3266 = )

Thanks,

JN
<= br>
= --Apple-Mail=_0601CF93-4FB8-43DE-9BAD-B9075B1001E7--