From nobody Sun Oct 02 21:36:42 2022 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Mgcjq04kHz4ddCQ for ; Sun, 2 Oct 2022 21:36:47 +0000 (UTC) (envelope-from maciphone2@googlemail.com) Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [IPv6:2a00:1450:4864:20::62d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Mgcjp0jhmz3kKS for ; Sun, 2 Oct 2022 21:36:46 +0000 (UTC) (envelope-from maciphone2@googlemail.com) Received: by mail-ej1-x62d.google.com with SMTP id rk17so18673777ejb.1 for ; Sun, 02 Oct 2022 14:36:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20210112; h=message-id:in-reply-to:to:references:date:subject:mime-version :content-transfer-encoding:from:from:to:cc:subject:date; bh=D4KimtIVCoKe4gKELOpFJDe6KpLjYyG/hzrCwAvNPEU=; b=LNEQGyI1ew2I+Z1vp7fQzgcJdZcwlkQjF5VfbeC1v2cPSAd1ace9MTLXYtSQ6yJPHo ag4+baFCgJRNtJox2XiW1snY3JjkVsKKgkZVWs5ItQjjMTPAdaXdq7062G5pLG32S48c 1GRY3zs4ZtMBVXhgEcJmsHiEA99+725Cv0wEfj8xFhbOhfQuU6sTNgpjTTBut1Qtt2Et JFeIl7Yb7wG3/Qab3JbI6/i4e3t5J2OnMcFfjkPVIQBmBIRZHf5jrIaRE0z1H3gKpH/x mgA/k6u5DIvmKdOSvT62jHd5k6jWay+P4RxjkM6ibS7ILLhA4BISLGmksMXoMXjBnVaf 4MWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=message-id:in-reply-to:to:references:date:subject:mime-version :content-transfer-encoding:from:x-gm-message-state:from:to:cc :subject:date; bh=D4KimtIVCoKe4gKELOpFJDe6KpLjYyG/hzrCwAvNPEU=; b=grlCz/EUXhqY6yi5X3kr82xtAONT6bFxFs01BGm2pHanY0/wRl7pCoromguooPLtN3 E3KOzC2EShMTyBQ1fwhWdWZrxEIw6rUq5BXJTYATzGD6j8uyY6z0SbhuiugGpIOmDa9s MmoRVR4hsXrj9L6Hjdi3/MC5rJsKW2S4UIVsQNPw9/PkxdG28JvsX9XS2t04tlGwCkZP CilUOWFTOh9E3qpeyCSY3x/nMgWQgjHTi7rblIp4VRs+JCTfrjlcPUTKAKLbvREewC2W I9UV8iUQAhZImXmzHvLxWyHcK6SSo+dWgi5eIo7wwykybQz64eRk9HlYNL96p39RnxDK 5fLA== X-Gm-Message-State: ACrzQf3O24z1+b2lpcaLcJnFPX6wdZXHB/4nxxprvwW9omD5ejtDfJGR UWB58BUTDizAH1lwIfVlVKBQv0jw+x50pQ== X-Google-Smtp-Source: AMsMyM49ldP5/ST7y4HBlWUjsCEcaVkxO3fZXJ3r08xAQud54pOT/y972zFl2Fj1zpY7rqgKa7f4eA== X-Received: by 2002:a17:907:9807:b0:781:feee:f87c with SMTP id ji7-20020a170907980700b00781feeef87cmr13500058ejc.101.1664746604669; Sun, 02 Oct 2022 14:36:44 -0700 (PDT) Received: from smtpclient.apple (dynamic-046-114-061-084.46.114.pool.telefonica.de. [46.114.61.84]) by smtp.googlemail.com with ESMTPSA id p6-20020aa7cc86000000b004574f4326b8sm6003024edt.30.2022.10.02.14.36.43 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 02 Oct 2022 14:36:44 -0700 (PDT) From: =?utf-8?Q?Klaus_K=C3=BCchemann?= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: Re: u-boot debug, was: Re: U-boot on RPI3, sees disk but won't boot it Date: Sun, 2 Oct 2022 23:36:42 +0200 References: <20220929002131.GA77106@www.zefox.net> <197D3C46-063B-4C67-AB1A-A3A072521D7F@yahoo.com> <6AA65AE6-41F1-405F-A592-7D641EA4C9CF@yahoo.com> <20221001174724.GA98055@www.zefox.net> <20221001193033.GA98348@www.zefox.net> <46226720-D867-4AD3-9559-A4365FAC28C4@yahoo.com> <6DB88FC9-629C-43E6-9673-32640FC547F7@yahoo.com> <20221002182049.GA2255@www.zefox.net> <5FFDAA6A-AD8C-4E40-A2EB-4082E5086679@googlemail.com> <38DFEB91-AC60-4FD1-8088-95B0A06C5E5D@yahoo.com> <64C1085D-D3F8-45A3-80FB-4B88F81E480E@googlemail.com> To: Mark Millard , freebsd-arm@freebsd.org, bob prohaska In-Reply-To: <64C1085D-D3F8-45A3-80FB-4B88F81E480E@googlemail.com> Message-Id: X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Rspamd-Queue-Id: 4Mgcjp0jhmz3kKS X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=googlemail.com header.s=20210112 header.b=LNEQGyI1; dmarc=pass (policy=quarantine) header.from=googlemail.com; spf=pass (mx1.freebsd.org: domain of maciphone2@googlemail.com designates 2a00:1450:4864:20::62d as permitted sender) smtp.mailfrom=maciphone2@googlemail.com X-Spamd-Result: default: False [-3.49 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.991]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[googlemail.com,quarantine]; R_DKIM_ALLOW(-0.20)[googlemail.com:s=20210112]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DWL_DNSWL_NONE(0.00)[googlemail.com:dkim]; RCPT_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-arm@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62d:from]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[googlemail.com]; TO_DN_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[googlemail.com:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[yahoo.com,freebsd.org,www.zefox.net]; FREEMAIL_ENVFROM(0.00)[googlemail.com]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org] X-ThisMailContainsUnwantedMimeParts: N > Am 02.10.2022 um 22:46 schrieb Klaus K=C3=BCchemann = : >=20 >=20 >=20 >> Am 02.10.2022 um 22:18 schrieb Klaus K=C3=BCchemann = : >>=20 >>=20 >>=20 >>> Am 02.10.2022 um 21:58 schrieb Mark Millard : >>>=20 >>> On 2022-Oct-2, at 12:35, Klaus K=C3=BCchemann = wrote: >>>=20 >>>> Am 02.10.2022 um 20:20 schrieb bob prohaska : >>>>>=20 >>>>> On Sat, Oct 01, 2022 at 02:21:42PM -0700, Mark Millard wrote: >>>>>>=20 >>>>>> http://nemesis.zefox.com/~fbsd/pelorus_console.txt7_orig_fragment >>>>>>=20 >>>>>> still shows all the debug output. It did not >>>>>> avoid the timing changes. >>>>>>=20 >>>>>> You might need to not use either of: >>>>>>=20 >>>>>> patch-common_usb__hub.c >>>>>> patch-common_usb__storage.c >>>>>>=20 >>>>>> and to disable the LOG_DEBUG and DEBUG lines in: >>>>>>=20 >>>>>> patch-common_usb.c >>>>>>=20 >>>>>> via turning them into comments by adding // as >>>>>> indicated below: >>>>>>=20 >>>>>> +//#define LOG_DEBUG >>>>>> +//#define DEBUG >>>>>>=20 >>>>>=20 >>>>> I think the changes were successful, u-boot compiles and >>>>> runs. There's no extra output, and unfortunately only one=20 >>>>> successful reboot so far. Bus scanning seems quite slow. >>>>> Storage devices are rarely found on reset, but usb reset >>>>> does sometimes work. Run bootcmd_usb0 paused for minutes >>>>> at Device 0: and paused again after reporting ..current device. >>>>> No echo from the console, ctrl-C did nothing.=20 >>>>>=20 >>>>> The attempt sequence was >>>>> SRBSPRMRPRRPUPPRRUPUCUUC >>>>> where=20 >>>>> S is shutdown -r >>>>> R is reset of u-boot >>>>> U is usb reset >>>>> P is powercycle >>>>> M is stop at mountroot >>>>> C is run bootcmd_usb0 >>>>>=20 >>>>> The console log is at >>>>> http://nemesis.zefox.com/~fbsd/pelorus_console.txt8_no_debug >>>>>=20 >>>>> It now appears that the run bootcmd_usb0 rather reliably gets >>>>> stuck, with the disk LED on steadily (no activity). Maybe in >>>>> one of the loops seen earlier?=20 >>>>>=20 >>>>> Thanks again for all your help! >>>>>=20 >>>>> bob prohaska >>>>>=20 >>>>=20 >>>>=20 >>>> So if you now reapply the #define DEBUG patches(while keeping the = mdelay-patch) and the reboot issues definitely went away >>>> we have a typical so called Heisenbug, hopefully more or less now = a fixed issue. >>>=20 >>> No. Bob has more than one problem: more problems observed >>> after "1 Storage Device(s) found". The DEBUG/mdelay >>> combination only seemed to cause the "1 Storage Device(s) >>> found" to be at least more reliable, not later stages. >>>=20 >>> It is not obvious if earlier activity contributes or not >>> to the problems observed after "1 Storage Device(s) found". >>>=20 >>> So far nothing has gotten near having things just work for >>> booting without manual intervention, multiple retries >>> being involved sometimes. >>>=20 >>>> Well, USB-boot problems on earlier Pi models( afaik all except the = 4) are commonly known, from defective HW to power cycle issues we will = find a lot of discussions on the WWW and we will see that even the = debug-message =E2=80=9Eis your USB cable bad?=E2=80=9C did fix issues = in some cases. Others applied RNG devices or external clock or even = plugging a mouse fixed it( to change usb enumeration). >>>>=20 >>>> I think with the working u-boot.bin after 1500 successful reboots = you can be sure it=E2=80=99s working =E2=80=A6. >>>> just kidding=E2=80=A6 :-) >>>>=20 >>>=20 >>>=20 >>> =3D=3D=3D >>> Mark Millard >>> marklmi at yahoo.com >>=20 >> hard to read and remember every log but I thought Bob wrote about = aprox. 30 successful reboots after the mdelay patch, >> while of course that could be coincidence, who really knows what = happens in this untrackable inconsistent behavior of the usb-boot?! >>=20 >>> Am 02.10.2022 um 21:48 schrieb Mark Millard : >>>=20 >>> (RaspiOS and Ubuntu do not use U-Boot last I knew. So >>> they do not make for good comparisons for the purpose >>> as far as I know.) >>=20 >> RaspiOS doesn=E2=80=99t , Ubuntu(and others) use u-boot since years = =E2=80=A6 >> while possible Ubuntu(or others) have own u-boot patches , >> from guessing it seems more probable that they also will sometimes = hang after (re)boot. >>=20 >> If I would want to keep such a device as an online server, like Bob = does, for whatever reason I would=20 >> Implement something like an =E2=80=9EIPMI=E2=80=9C or simpler said: >> An immediate console remote access after being warned by a script = that the machine is offline. >> But I would remove it from cluster if there are known Hardware = problems.=20 >>=20 >>=20 >> Regards >>=20 >> Klaus >>=20 >>=20 >=20 > =E2=80=A6 but of course, Mark, that is correct : > overwriting parts of the msdos-partition by linux ones could be the = last resort to save something=E2=80=A6 > but if linux had patched inside u-boot, as you did it for Bob, I would = see other problems coming=E2=80=A6 >=20 > Regards >=20 > Klaus >=20 Not debugging related, so off-topic: Bob, if you care about remotely rebooting without losing access to the = console: You can plug a known functioning online machine(e.g. your Pi4) v ia GPIO = directly to Pi 3b. ssh into the Pi4 in this example , then cu / minicom /xy into the 3b = console and manually reboot the 3b over the u-boot console if it hangs. For unexpected 3b-crashes while uptime, as said, you could use a warning = script or whatever. Seems to be easier this way than continue hunting an untraceable bug . But (at least my) main intention was to come closer to the u-boot source = code, So many thanks to you,Mark ,giving so much POWER(and SPEED) to these = things, great work! /Bob knows what I mean with POWER & SPEED terminology :-) Regards=20 Thanks=20 Klaus=20 =20