From nobody Wed May 31 19:52:55 2023 X-Original-To: wireless@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QWg1K2sxLz4XW2l for ; Wed, 31 May 2023 19:53:25 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from webmail5.jnielsen.net (webmail5.jnielsen.net [IPv6:2607:f170:34:11::b0]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.freebsdsolutions.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QWg1K01bjz3hGf; Wed, 31 May 2023 19:53:24 +0000 (UTC) (envelope-from lists@jnielsen.net) Authentication-Results: mx1.freebsd.org; none Received: from smtpclient.apple (166-70-14-17.xmission.com [166.70.14.17] (may be forged)) (authenticated bits=0) by webmail5.jnielsen.net (8.17.1/8.17.1) with ESMTPSA id 34VJrBcR032745 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 31 May 2023 13:53:14 -0600 (MDT) (envelope-from lists@jnielsen.net) X-Authentication-Warning: webmail5.jnielsen.net: Host 166-70-14-17.xmission.com [166.70.14.17] (may be forged) claimed to be smtpclient.apple From: John Nielsen Message-Id: <5264D290-EB82-4D24-A812-85E3E6B5C88E@jnielsen.net> Content-Type: multipart/alternative; boundary="Apple-Mail=_C8BA0CC5-1187-4926-8F88-FF63B1972285" List-Id: Discussions List-Archive: https://lists.freebsd.org/archives/freebsd-wireless List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-wireless@freebsd.org X-BeenThere: freebsd-wireless@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.600.7\)) Subject: Re: Help me grok the ath(4) device attach code Date: Wed, 31 May 2023 13:52:55 -0600 In-Reply-To: Cc: "Bjoern A. Zeeb" , "wireless@freebsd.org" To: Adrian Chadd References: <49AEA1CB-FA85-432F-89D7-8C49B5F3A344@jnielsen.net> <85BF3AB2-2EBF-4398-A507-ABA35505A56C@jnielsen.net> X-Mailer: Apple Mail (2.3731.600.7) X-Rspamd-Queue-Id: 4QWg1K01bjz3hGf X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6364, ipnet:2607:f170:30::/44, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_C8BA0CC5-1187-4926-8F88-FF63B1972285 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On May 30, 2023, at 11:17 PM, Adrian Chadd wrote: >=20 > On Tue, 30 May 2023 at 22:12, John Nielsen > wrote: >>> On May 30, 2023, at 10:56 PM, Adrian Chadd > wrote: >>>=20 >>> On Tue, 30 May 2023 at 20:56, John Nielsen > wrote: >>>> > On May 30, 2023, at 8:02 PM, Adrian Chadd > wrote: >>>> >=20 >>>> > Err, if it's coming up w/ that MAC then it's not finding and = attaching right to the OTP/EEPROM calibration information. That's the = big red flag that it in general won't work correctly. >>>> >=20 >>>> > Can you provide the rest of the ath_hal messages? I'd like to see = what it's saying during boot around it checking the EEPROM/OTP contents. = It's possible there's some work around required for this NIC. >>>>=20 >>>> He speaks! Thanks for taking the time. I just realized that = ath_hal_printf doesn=E2=80=99t prepend =E2=80=9Cath%d=E2=80=9D so I=E2=80=99= ve been missing those messages when grep-ing. Here=E2=80=99s the whole = snippet: >>>>=20 >>>> ath0: mem 0xf7a00000-0xf7a7ffff at device = 0.0 on pci4 >>>> ar9300_flash_map: unimplemented for now >>>> Restoring Cal data from DRAM >>>> Restoring Cal data from EEPROM >>>> Restoring Cal data from Flash >>>> Restoring Cal data from Flash >>>> Restoring Cal data from OTP >>>> ar9300_eeprom_restore_internal[4338] No vaid CAL, calling default = template >>>> ar9300_hw_attach: ar9300_eeprom_attach returned 0 >>>=20 >>> Yeah, this bit right here is the problem. It's not finding a valid = calibration. >>=20 >>> oh err, is there a wifi enable/disable switch or something? maybe = it's asserted and somehow it's mucking up the NIC? >>=20 >> There is a physical switch and it=E2=80=99s in the =E2=80=9Cenable=E2=80= =9D position. >>=20 >>> I wonder what ath9k is doing here? Is there some weird pci based = workaround/flag for the given NIC PCI id? >>=20 >> That was the first breadcrumb BZ threw me but I can=E2=80=99t find = anything. There are some .driver_data hints for adjacent subdevice IDs = but none for this one (Dell 0x020d) in either FreeBSD or Linux that I = could find. >>=20 >> The kernel on the Arch Linux USB I have handy doesn=E2=80=99t appear = to have been compiled with CONFIG_ATH_DEBUG but here=E2=80=99s what it = has in /sys/kernel/ieee80211/phy0/ath9k/base_eeprom: >> EEPROM Version : 2 >> RegDomain1 : 108 >> RegDomain2 : 31 >> TX Mask : 3 >> RX Mask : 3 >> Allow 5GHz : 1 >> Allow 2GHz : 1 >> Disable 2GHz HT20 : 0 >> Disable 2GHz HT40 : 0 >> Disable 5Ghz HT20 : 0 >> Disable 5Ghz HT40 : 0 >> Big Endian : 0 >> RF Silent : 45 >> BT option : 0 >> Device Cap : 0 >> Device Type : 5 >> Power Table Offset : 0 >> Tuning Caps1 : 0 >> Tuning Caps2 : 0 >> Enable Tx Temp Comp : 1 >> Enable Tx Volt Comp : 0 >> Enable fast clock : 1 >> Enable doubling : 1 >> Internal regulator : 0 >> Enable Paprd : 0 >> Driver Strength : 0 >> Quick Drop : 1 >> Chain mask Reduce : 0 >> Write enable Gpio : 6 >> WLAN Disable Gpio : 0 >> WLAN LED Gpio : 8 >> Rx Band Select Gpio : 255 >> Tx Gain : 1 >> Rx Gain : 3 >> SW Reg : 303972983 >> MacAddress : 44:39:c4:5b:44:4a >>=20 >> It also has some calibration and other data in modal_eeprom.=20 >>=20 >> There is this commit in ath9k which mentions an alternative EEPROM = address, but I=E2=80=99m not sure if that=E2=80=99s relevant. =46rom = what I can tell the probe should succeed at the normal base_address = 0x3ff instead of needing to try the =E2=80=9C4k=E2=80=9D one 0xfff. >> = https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git/commit/drive= rs/net/wireless/ath/ath9k?id=3D528782ecf59f7bab2f1368628a479f49be59b512 >=20 > Yeah i'd try that. It'd be nice if I knew that the NIC used OTP or = EEPROM though. >=20 > There's known issues with all the Atheros chips (sigh) with how the = EEPROM and PCIe bus reset .. interact. > (If the bus reset is too short then the EEPROM state machine gets = stuck and nothing gets read.) It makes debugging this hard because the = NIC itself will work in another device fine, because it's the BIOS/ACPI = code. :( The 4k EEPROM read didn=E2=80=99t work. But I did notice that where it = says it=E2=80=99s restoring Cal data from OTP it actually just does an = EEPROM read again. Shouldn=E2=80=99t this line be a call to = ar9300_otp_read() instead of ar9300_eeprom_restore_internal_address()? = https://cgit.freebsd.org/src/tree/sys/contrib/dev/ath/ath_hal/ar9300/ar930= 0_eeprom.c#n4292 Otherwise it doesn=E2=80=99t use the 0x14000 (0x30000 for some cards) = OTP offset as a starting point. -John --Apple-Mail=_C8BA0CC5-1187-4926-8F88-FF63B1972285 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
On May 30, 2023, = at 11:17 PM, Adrian Chadd <adrian@freebsd.org> = wrote:

On Tue, 30 May 2023 at 22:12, John = Nielsen <lists@jnielsen.net> = wrote:
On May 30, 2023, at 10:56 PM, Adrian Chadd <adrian@freebsd.org> wrote:

On Tue, 30 May = 2023 at 20:56, John Nielsen <lists@jnielsen.net> wrote:
> On May 30, 2023, at 8:02 = PM, Adrian Chadd <adrian@freebsd.org> = wrote:
> 
> Err, if it's coming up w/ that = MAC then it's not finding and attaching right to the OTP/EEPROM = calibration information. That's the big red flag that it in general = won't work correctly.
> 
> Can you provide = the rest of the ath_hal messages? I'd like to see what it's saying = during boot around it checking the EEPROM/OTP contents. It's possible = there's some work around required for this NIC.

He speaks! Thanks = for taking the time. I just realized that ath_hal_printf doesn=E2=80=99t = prepend =E2=80=9Cath%d=E2=80=9D so I=E2=80=99ve been missing those = messages when grep-ing. Here=E2=80=99s the whole snippet:

ath0: = <Atheros AR946x/AR948x> mem 0xf7a00000-0xf7a7ffff at device 0.0 on = pci4
ar9300_flash_map: unimplemented for now
Restoring Cal data = from DRAM
Restoring Cal data from EEPROM
Restoring Cal data from = Flash
Restoring Cal data from Flash
Restoring Cal data from = OTP
ar9300_eeprom_restore_internal[4338] No vaid CAL, calling default = template
ar9300_hw_attach: ar9300_eeprom_attach returned = 0

Yeah, this bit right here is the = problem. It's not finding a valid = calibration.

oh err, is there a wifi enable/disable = switch or something? maybe it's asserted and somehow it's mucking up the = NIC?

There is a physical switch and = it=E2=80=99s in the =E2=80=9Cenable=E2=80=9D = position.

 I wonder what ath9k is doing here? Is there = some weird pci based workaround/flag for the given NIC PCI = id?

That was the first = breadcrumb BZ threw me but I can=E2=80=99t find anything. There are some = .driver_data hints for adjacent subdevice IDs but none for this one = (Dell 0x020d) in either FreeBSD or Linux that I could = find.

The kernel on the Arch Linux USB I have = handy doesn=E2=80=99t appear to have been compiled with CONFIG_ATH_DEBUG = but here=E2=80=99s what it has in = /sys/kernel/ieee80211/phy0/ath9k/base_eeprom:
  =     EEPROM Version :         =  2
          RegDomain1 :   =      108
          = RegDomain2 :         31
    =          TX Mask :         =  3
             RX = Mask :          3
    =       Allow 5GHz :         =  1
          Allow 2GHz :   =        1
   Disable 2GHz HT20 : =          0
   Disable 2GHz = HT40 :          0
  =  Disable 5Ghz HT20 :         =  0
   Disable 5Ghz HT40 :       =    0
          Big Endian : =          0
      =      RF Silent :         = 45
           BT option :   =        0
        =   Device Cap :          0
  =        Device Type :         =  5
  Power Table Offset :       =    0
        Tuning Caps1 : =          0
      =   Tuning Caps2 :         =  0
 Enable Tx Temp Comp :       =    1
 Enable Tx Volt Comp :     =      0
   Enable fast clock :   =        1
     Enable = doubling :          1
  Internal = regulator :          0
    =     Enable Paprd :         =  0
     Driver Strength :     =      0
          Quick = Drop :          1
   Chain = mask Reduce :          0
  =  Write enable Gpio :         =  6
   WLAN Disable Gpio :       =    0
       WLAN LED Gpio : =          8
 Rx Band Select Gpio = :        255
        =      Tx Gain :         =  1
             Rx = Gain :          3
    =           SW Reg : =  303972983
          MacAddress = : 44:39:c4:5b:44:4a

It also has some = calibration and other data in = modal_eeprom. 

There is this commit in = ath9k which mentions an alternative EEPROM address, but I=E2=80=99m not = sure if that=E2=80=99s relevant. =46rom what I can tell the probe should = succeed at the normal base_address 0x3ff instead of needing to try the = =E2=80=9C4k=E2=80=9D one 0xfff.

Yeah i'd = try that. It'd be nice if I knew that the NIC used OTP or EEPROM = though.

There's known issues with all the = Atheros chips (sigh) with how the EEPROM and PCIe bus reset .. = interact.
(If the bus reset is too short then the EEPROM state = machine gets stuck and nothing gets read.) It makes debugging this hard = because the NIC itself will work in another device fine, because it's = the BIOS/ACPI code. = :(

The 4k EEPROM read = didn=E2=80=99t work. But I did notice that where it says it=E2=80=99s = restoring Cal data from OTP it actually just does an EEPROM read again. = Shouldn=E2=80=99t this line be a call to ar9300_otp_read() instead of = ar9300_eeprom_restore_internal_address()?

https:/= /cgit.freebsd.org/src/tree/sys/contrib/dev/ath/ath_hal/ar9300/ar9300_eepro= m.c#n4292

Otherwise it doesn=E2=80=99t use the = 0x14000 (0x30000 for some cards) OTP offset as a starting = point.

-John

= --Apple-Mail=_C8BA0CC5-1187-4926-8F88-FF63B1972285--