From nobody Sat Feb 12 21:32:25 2022 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 780961957849 for ; Sat, 12 Feb 2022 21:32:37 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-21.consmr.mail.gq1.yahoo.com (sonic309-21.consmr.mail.gq1.yahoo.com [98.137.65.147]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jx3c440vWz3q17 for ; Sat, 12 Feb 2022 21:32:36 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1644701549; bh=tGUZy8QXjEFyNiXMeCyCzPGBHqhq1udZ4LQJfi6nNn0=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=eWYaSi2/gZ690zkLepMGFP0ncqPYpKpECrVAxIBHlCBk1+uhUInl81pj7nc6xJtwJZmpcSwoO9IwNGOeRr02gwgwwZThfpZskIkYFl0S9GWiRTHEMDMYvv2G7iHalc6D93OsxrH1bVgUb8mp9qPm4khVsR9nf/T5gte/1r9OqoPNd3mN2Zc2vHZDFoXMH/FoJQB9fle9zd26/d2mr2vtG+IEPwMqPqrxSpMS8+jZG87Uq36EFFM2bjU3wHD2Tt+9TqRG1pZ60jfZlLKunef00C0nW+ClUFcpMtioWFp4WlnvuZQBJ7SQ7ARaYD1Twhk65zEVSidJ+SkOFgaF8W8qCQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1644701549; bh=kJdt0WKLtIfgTMLom26e8t9SKPejTKc3m5ND0VS2QoJ=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=AMUvQBQTGfznhg6phoZpdzXZP2THQFzVhj0+0XPTAU1i20TrqjCzbiqGQoJ1y9urgQwTBTzXlhkLyZFsAq6Gr+D1yHoJYmsQHd9EwkLEAkvKi7BZ4qTTn2Ekm/OLCdJkaV6NtqAMEi6gpJKZ1/djdAvk1QaJHX4mhtEQYIaMsMOIWb5WHGmeQYRhOZy7AT2fKpDqBvNWnMBggkFAavZ1d6yeQhRK6++GIfu9613c5SIqPi/Cu7nGgGn8RlHz5fxlSdUv980/+ZPUq4lIB6QpXf5IVyUfCmRpLnrCxTN2f6MCPGgCRg3grL+9DpRAOCRBAS7esXOrFrolNujMPn7WiA== X-YMail-OSG: l8Wmo0sVM1lXPIgmUN9K2B.S2l3Yj_CDDJpB4c1docVRElDkZhetVjRUPPvxpPk 5FR5FoQnCFMP2AC3oS9csBdINJOrYm9QO0rAzL_XoDy_3FT.xE1qEqT1xeclkBQ1t60rcohvDgd8 qRcMNYl702EPuczY_W1WfuIzQV.WaYYxoN47cYyT5_HEUZnNliX0yjCw9e_Mkp5KyyxvtAa8Ner5 yuHdQwjmygzzDzI6QPYYqOc48yAgsYKETusXQl8x.2ukkQLIcdtXj2GcJ61mc87JrpIia_fUCle3 Yr4qms2XctW6zmklbE.6EIkQ6p41jd77HlhZ_nEhX6KNR4uP5X7443oRsAYmlm7grNHA_8uJTgAL Fo7RLz5R0fPuQgEzJto6y06P4A7IdFQaMSGFQmfbXgDSa87c.Tj1S64SL21oEKj0N6ZA5uaEVOqK nsXRLXGbmtDsH5JEUuDCkKoRWaNfBcr34Q9AELqtA_fUDV6ZMUbZr6JukjgvCs3gJFC6LHybMhGI .iKioyVhGbElCTNCtorSINWKv1mK6OgUw8cT4yaV0mk37Rm5qMutqgq.L3XJ3y8W64Bxk7.QTDCd Wd4kdn07GiG.wJ2P4lBL.DEVCmErEWCND0DCn9Ni58D5KgxTJG5Uts.Fv5ycVSlgzMVfhiJtYJ5T UG_6zFw0t6Th2q2Jtgh0w1x2OMefcK4tGPmqcJv9iOylXiF41JkDGD7j_btXzMRMkmJByNb1sQja jjxIkfQoaWK.1XoDJuXcHBTRP3tXekayP0hmmeaBMCJllgtXYa6nNiRLmvSsNiBfTCuUm2vgdwzV EQaZz4Lsnk1oiggPkpZVNtlrXk3xHBmCXrKgMuMYeOpk1xXBLwHwv8yEffF4HLVOID1qBpmQdUTx RMPcB_1.RApEgXXamGH5GOZnPvXF3UawonaZ5G33RwtR67w.GI68aEB_W51t.t70pVz4pM.FOgRx C41Ar9wniHRhhCIUJ9ewAA0ctHNCbayEI1za1GRw0fUMCIH7s7ImwQoIGT9znjJeWz9eB9MaDI5B bR9hyfQcniYkVEHWcH94NeG51MVglc48GAFe8aIpU4SZw2yCfOycJs5iHVY5Eux.KeMvwjNNbINL w4dWUMI.5wTUYjggBtL8rYF1jMeRzqPP_B044aviSfk4zvwNMXiPORz5sm3xHWWne.MjQJobb8Xo EJJDomWYutQMJUleE0nbD2S2bghKTAaXIMFZ9KTlyfzqPSsvY0YHedotebkB5bWiOgbcG_Mv3pk. Zu1DTvxO.CLC5_TjYVomBpbPB2gFimWPjESTMClPwAmCG99ujucQ8kfhqIOSjl9O9mq4y4G6..AH rXkAo_Q.vhSHm3sBJNBC5DWQLpYcak53tSD7WlsuhG8Q7t.OCVvALkcWfZzlBCJEvDsBhZPYV3o_ PHSgjMkjJ9aLH_MMCwwxZx9m2QVdUMpenwwFoYHFFx6kY.NQ05gWGFWBw59rHNfMilbnW14UR5WO AMiMiC9w.OyIrGV7.bYLYmEDatVWs9rGyXs0tal9qwx7RUrk9E.8JATQxe6gHpAYbD_C2stSWcTS 0myAF93Dl3mXa9vwOweHLcP3aclWsfXi_5CzxoCAqb7tIChoQ.0SDz3dLFFgV0R5YkZiZmjsSQAD 6JnCAaor4hQEijS5vRKeVFN9rbb1BjM9tX_LCC2rt45T33dQcq4IM1vCJoef1fISoKuB27zd3Cbg xV7eYCn0k1_83ioxWhzFIdCC74rTTTSXKcO5It_2Hc1LkGTLnVGGrzT8haQXW8NEg83OinuK34VQ Vl4edS2eT1Eu8u.XNdZqSTiqs6Gtb6kg9Cbtv.x75jCLSURSQ6tgY_iZtq50Q3V5BiHD.8PGPefY jrGgyvvwmAJYC8aA.gSvcJSxq9B7bVT5xL3hwRfKJKB06q.ymLbpo8YOvmRaGSk6PcJu7rrTr0YB zpKCTmou34deFyX2uxpBc7UOpUJKB.OQ_MrVQThYR.SgtonBDDY1qZ8_RVksPMN5PPn2ymW3VzV_ z_iI6A_XsM74iJ2GYserUF8u5zxlqJG8txOfvS013JejGu1OawOkYrIpefsA4J00fAEGPup29t8Y _YXp017i4hVHUeWYk1st.ANStj.7m_sQssYufs4tRoHG10XOcKqkCKtpQ.BOkB4xrajkgkvfwIKh BAClxIayB1pzqElFXv65Vcf.MgrfZwyZnxzRMd0z_PpkvrY7ZbqHkOPL270uHzfwv_qTJJGPz0au IxiWxJYApWo1V0EhwHeo7t03UPT6mhUV0SmvmZx3nYu6fWk3NXSoHxyquZuLkRjt0MGh.q6aG68Y eUyQHgPKdUykXIwxIOE9Lv6Dwkid9mDOAc8aCUlhn22E- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Sat, 12 Feb 2022 21:32:29 +0000 Received: by kubenode513.mail-prod1.omega.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 2026682897702a4b34e84159cbb72ff1; Sat, 12 Feb 2022 21:32:25 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: Pi3 answers ssh only if outbound ping is running on -current From: Mark Millard In-Reply-To: <20220212185618.GA37391@www.zefox.net> Date: Sat, 12 Feb 2022 13:32:25 -0800 Cc: freebsd-arm@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <34F5C092-7C35-4CBB-9CC3-99E373D1F785@yahoo.com> References: <20220212185618.GA37391@www.zefox.net> To: bob prohaska X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Rspamd-Queue-Id: 4Jx3c440vWz3q17 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b="eWYaSi2/"; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.147 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-3.50 / 15.00]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-0.997]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.147:from]; MLMMJ_DEST(0.00)[freebsd-arm]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.147:from]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N On 2022-Feb-12, at 10:56, bob prohaska wrote: > For a few weeks now a Pi3 running -current will not respond to > an incoming ssh connection unless an outbound ping process is = running. >=20 > Once the outbound ping is started via the serial console, incoming > ssh connections are answered normally. Uname -a reports > FreeBSD www.zefox.org 14.0-CURRENT FreeBSD 14.0-CURRENT #10 = main-n253073-6db44b0158c: Sat Feb 12 04:30:21 PST 2022 = bob@www.zefox.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64 >=20 > A Pi4 running -current of a few days ago exhibits no such problems. >=20 > Another Pi3 running stable/13 has been behaving in the same way. >=20 > Both Pi3s successfully set time via ntp on reboot and will > very briefly (one or two minutes) prompt for an ssh password, > but no further progress is made and the login attempt times out. > If the ssh login is attempted a second time, not even a password > prompt comes back. >=20 > Ping times (to an adjacent machine on the same subnet are > 64 bytes from 50.1.20.26: icmp_seq=3D2 ttl=3D64 time=3D0.978 ms > 64 bytes from 50.1.20.26: icmp_seq=3D3 ttl=3D64 time=3D0.967 ms > 64 bytes from 50.1.20.26: icmp_seq=3D4 ttl=3D64 time=3D1.088 ms > 64 bytes from 50.1.20.26: icmp_seq=3D5 ttl=3D64 time=3D0.983 ms > 64 bytes from 50.1.20.26: icmp_seq=3D6 ttl=3D64 time=3D1.007 ms > 64 bytes from 50.1.20.26: icmp_seq=3D7 ttl=3D64 time=3D1.075 ms > 64 bytes from 50.1.20.26: icmp_seq=3D8 ttl=3D64 time=3D1.020 ms > 64 bytes from 50.1.20.26: icmp_seq=3D9 ttl=3D64 time=3D1.044 ms > 64 bytes from 50.1.20.26: icmp_seq=3D10 ttl=3D64 time=3D1.026 ms > 64 bytes from 50.1.20.26: icmp_seq=3D11 ttl=3D64 time=3D0.908 ms >=20 > That might be considered slow, but the correspondent machine > is only a Pi2 running=20 > FreeBSD www.zefox.com 14.0-CURRENT FreeBSD 14.0-CURRENT #3 = main-71d2d5adfe: Tue Dec 21 00:23:51 PST 2021 = bob@www.zefox.com:/usr/obj/usr/freebsd-src/arm.armv7/sys/GENERIC arm >=20 > If the outbound ping is started, an incoming ssh connection = established > and the outbound ping subsequently stopped the running ssh connection > silently freezes; no disconnect, but no response, not even echo. Some > tens of seconds later, all inputs were responded to. Tried a second = time, > the stoppage recurred, restarting the outbound ping eventually = restored > responsiveness. >=20 > With the outbound ping stopped, an inbound ssh attempt silently = failed: >=20 > bob@raspberrypi:~ $ ssh -vvv 50.1.20.28 > OpenSSH_7.9p1 Raspbian-10+deb10u2+rpt1, OpenSSL 1.1.1d 10 Sep 2019 > debug1: Reading configuration data /etc/ssh/ssh_config > debug1: /etc/ssh/ssh_config line 19: Applying options for * > debug2: resolve_canonicalize: hostname 50.1.20.28 is address > debug2: ssh_connect_direct > debug1: Connecting to 50.1.20.28 [50.1.20.28] port 22. > [enter key echoed] > debug1: connect to address 50.1.20.28 port 22: Connection timed out > ssh: connect to host 50.1.20.28 port 22: Connection timed out > bob@raspberrypi:~ $ =20 >=20 > Thanks for reading and any insights. If I've omitted useful=20 > details or tests please indicate. >=20 You have made multiple reports to the arm list for this issue without anyone having managed to help. This report does have more comparative context, which might help someone help. It may be time to try other lists like freebsd-net and, possibly, freebsd-hackers or freebsd-stable or freebsd-current . However, the best thing no matter where you go would be to (approximately) bisect toward the back-to-back FreeBSD version-pair on, say, stable/13 at which the the problem goes from not-there to happening. ( stable/13 changes slower and so has fewer versions to deal with. Also its KBI may grow but is constrained to otherwise be more stable [ relative to releng/13.0 ]. So you are less likely to run into version compatibility problems for the below suggestion.) I'd recommend using kernel and world materials from: https://artifact.ci.freebsd.org/snapshot/stable-13/?C=3DM&O=3DD on a separate microsd card updated from a normal context, avoiding builds. Remember that older stable/13 worlds can run on newer kernels generally. So you might only need to update the kernel after getting an initial, somewhat older context in place. (It is not obvious if it is a kernel-only problem or not.) If it is a kernel problem, you might be able to put down a releng/13.0 world and never change it during the approximate bisect activity. For what https://artifact.ci.freebsd.org/snapshot/ has available, this avoids having to build the versions. It also allows checking if your builds are behaving differently than the official snapshots do. https://artifact.ci.freebsd.org/snapshot/ may not be able to get you to the back-to-back FreeBSD version-pair: the range might be wider. Sometimes the wider range is enough by inspection of the types of commmits in the range. So I'd report whatever range you find wihtout having done any builds. I'll note that I have no problem with connecting via ssh to a RPi3B running my build of (line split for readability): # uname -apKU FreeBSD Rock64_RPi_4_3_2v1p2 14.0-CURRENT FreeBSD 14.0-CURRENT #28 main-n252475-e76c0108990b-dirty: Sat Jan 15 23:39:27 PST 2022 = root@CA72_16Gp_ZFS:/usr/obj/BUILDs/main-CA53-nodbg-clang/usr/main-src/arm6= 4.aarch64/sys/GENERIC-NODBG-CA53 arm64 aarch64 1400047 1400047 I have no stable/13 context set up for a RPi3B, only stable/13's that have an untuned ZFS context. Still, I wonder if that might operate well enough to test the issue, despite the 1 GiByte of RAM limitation. I may test that later today. =3D=3D=3D Mark Millard marklmi at yahoo.com