From nobody Sun May 14 20:00:46 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QKCzz1wHWz4C4FB for ; Sun, 14 May 2023 20:01:03 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic313-20.consmr.mail.gq1.yahoo.com (sonic313-20.consmr.mail.gq1.yahoo.com [98.137.65.83]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4QKCzy1Sdvz3l6X for ; Sun, 14 May 2023 20:01:02 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=KP6ZZyRX; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.83 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1684094460; bh=juiosNCkuc4fraPLKnWWkZZdg9KR4UqzbAgoMV4iraI=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=KP6ZZyRXansnYD5o3nDCH3VfHYjzTW97R11OkO94YE243QtVSQtUmuTjNi7kwHpAA4jwJtDdNLmDnOB1bpQGNbIQ+rAQ4uc1hWR4vnhC8Ki8ErSiZSSbnJ/UWFtSc4ahXfvLCxcMaZbM+egVDqag2890c2cNKCAyvEml9c036NFwS7E1++fhPeBUb0sNkNSlG9h0tZ6PALRfei7XbMC8v3LkUwA/oOCh66I0IgRcv8BNimjQVCezY1HnTYzKSuWl5jlK2kI2ur74bRNxQq8n1Np7xwF0euJbWC/hmE5LTbrt6bwzloNgk0ZJTfZ+AA1T+RYCyL3Bg+BlTsRa4KjlBA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1684094460; bh=/U1UlJeomYjiG4IxPzWVSRJuiCfgxNL7gSZtjHaHeKI=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=r1MF4YJtOyqrDM1M3dT07TXlbsFBz7SQajiDylZR4IPw+ZO54l4xkP86DQ9DrCE9JrpqsoL3p5R+zOfMnAV5CQEOVCyYJa3kxhyKhOyMcF3escB/qKYDK0ArOIp6ZJ6ER3DwehVxXImLvy8MPi1TY+LkGVboDcYrxDFAYOkX9fYbW4+PTKZDnMuxEAFJAUWZxjJRRIP4lXjTaOljrQyi2+KSQNU9QMBMi/0aYqyXrnmDevcay0yE9qhjrS4AEk6NY7dyQV80CHGzpwav7tYlS4xR9v67ObF3ma/7iwpCJh4469+8ZnQKnjf/wQn9TggS+MsR21PKhKaINju2bDDBZw== X-YMail-OSG: 12E.YFkVM1n28JUbIVv9Ja8SEeB61MDZnfUP8RgdTlqbAOE9WfKFt3YNv74NMpG dkdiAR0oxqf0gjOrjsgFNY4a3aFolL8Zh9CbWfUdxGS7MUXqkyjgb75rDOQzZ37QILuazfxS0sBY zw9SDuOTplCMlVJd0vMimfWNRp_DcC1NTilMNOrSzrdt8hnm96_d75dp0DGXNHLZh6gffSMjQpG8 lLOixe7rdgHqU5G0S0.pgMk3JwcvHFBN5ond2473.V53Qmyq86areZVE3EpMGlIOIe4Qoair3AvU 6dn4wJki_tvcIoFGVOgvfra5GK9B_sOAyyNm_6dEgoK.eGC.iphZxVeZ9UF5A4DEaIstTJRxM_8f hFgeFf144qHgKvDpMMQXJlerpvEpFL.sijn7c_N3.NTY_BAtHapeUb5DSvsbWa4nabu7XdMlHNDq gqpdZ41qVRTL2w592qX9WMgPb2WSeB_N7jIgl12zW22ZeNNBHfmgmqZdVWy8JtjUYg9hUGf6v6jA _yyrokIuVLYAwH0W_bqKxU0uSmwJdEVCNn5E0392owNolKCiA0o3KcZzeBPF2HkJB55dtdHEyRIr N0uUK1vikrPheYX7vBMdIctqWPFA8NDA3IwEnDx5FVHI_tARY32ZReR7gzxebqPS900onFF0P3Tr d_1niNQq1ME2Q_NbEgtFZadMllKj6mcTK8FMZ6vPQIIiBKH5D_hVJlYzGb1WH2TlY55HOGEBg8r4 Bz_6ixrk_.kwUSUV1dp6lkbdEgI.iqyIoU0zLDl5QaVxZGbU3oKs0UtQtBocgMoz8JuUq493.rrz WTXBA92N8pStMpUzf7W126uyMKrG7EJeEZqGo89lFzDRWsMaq51TvIHZxcdrDEJ78Pfl1jrImZRU mhL_eu_JOc2ldxGygXdK7hgL9x0Jh3K0.sT7JhVFh3aiE5xMN1bo38SaqCElcgwnmLnTGLbCqyb3 W7Zt4pWxprzg5myOG7.QeKAdQ6GWw2pUOV44MgusbFUKZ_2HOHOMnlqGxZHqcsJqZ_9a4qFT4yp7 tMX5zEEdy.eku9RrL56R7JTTIj_WELWzjDDidWmHjUp6YX_K_hrtUUcSS6hdYwAZ6UPi0HYA1IPc bpRrUueYntoaefxeKlpJ191pUOtEbW8R_LjrIHhe1MkUFPiFyGZtTJPR4mrnlcDt1G.83JFzGz0j V5UAZbRzlYr.1FwbzEqB2friqQp_olcz4psUrxNzKEx3k91n6RV85HsM3VMRS1UFyQiodEy97DLD UjLXZjHZBk2vX3nXP4Vsq2vu.G6pk.La42YDM1VQCE9HcfP.eMURyWDXrNHuUq1A1Wwf6iWN1ON9 k6Gs_HXkOwdrAaF9Z1dsVO5.udMY1I8OE.9HYqw0hJuph5Fk0QdC2U8QMgx7hgkckTUFHHzr20_U KhIqo__FW5ux7ESjMh4BqFbvgxtljZQ1by4f1UIkF54Z3xqL.P67WCtVXVBJxCkjd6.AfvvwwDqH IsanKDCaHR.6Io9B.lp.57YPYTsYX3wh1Tw3tpSZxa6IeJ1DV_5x9OAQ0NKWvaAJErgs.CN3rk2I oCQ6QNr8k2cxPgxI2zkz4to5hBep4wpZCvKCrEip9J9GA.f.FJOCnbRE45QeBdJv8AAOWNmS3kjk v4qCvjsvE5NYIFkyANbi7v0tENGBgACcE8gW0nOj84Uu3UWkevrkgYVF.jkSkGRXJd8uD7is0MEJ bsqTd3UBeRG5W6vYPmYYMBaHN0tUg.HA3fHAXGdxwE7qTUwvAcVWW9RGDPXvgFCWu8LIBuI031e6 oYoO.sguTNZDYqK84AAemSbvubnxSfiEZqfrnuiVgcB__HD1LcNvmFftDC9B_S6d608djZEjuFVn JPmtuZEARds8V4FzwXk936O9uapnq2H3JgKsxIKcYruxlCPbAgMKKfylFURjXlx5KlrYrhuLlPco 62hNSSHA3061BVp14Lr2cCuljWmxmeOV7hvhXJXe7Q6iMWXW6ySagh2S.W3K9E9k2nDUkiwZsPfq puI6Qgl5WQmPmeJUcY9R9EkHraIUDZQQuX1Dnk0ZFMvIVW5uNRW30pWBE_XCkBlOMvfYJBt2DktW b6Xa2kZ7nru.hkP6RoL1xdTeh9sHuL2q_lcIr0GRTbUcdCqsSsfSL2zwbYpWNTOL006kYcPc_.m1 LxQ3L2y8XVJsMZ5d1ZRCbOKXhJGjJ9q1vUWsP0Sho4DFbOq.2LpDRDoQVCe.iVXpMlA7KVrx.Be7 OqGMwUw2TNx7WzQ4yxtMaW6W_xqaMWesdUtAJKDlIf9m1Wgl9mPe7rxwUysmgimdRa1PqoI2BYZr 9 X-Sonic-MF: X-Sonic-ID: 10ef458e-3114-4bcb-b30d-93c05309464c Received: from sonic.gate.mail.ne1.yahoo.com by sonic313.consmr.mail.gq1.yahoo.com with HTTP; Sun, 14 May 2023 20:01:00 +0000 Received: by hermes--production-gq1-6db989bfb-js4gb (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID ef7f8a54bbff0ae6b991f6ebab42db92; Sun, 14 May 2023 20:00:57 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: Armv7 (rpi2) getting stuck in buildworld for -current From: Mark Millard In-Reply-To: <7758885B-3115-47F0-A453-1C010313D4B8@yahoo.com> Date: Sun, 14 May 2023 13:00:46 -0700 Cc: freebsd-arm@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <7758885B-3115-47F0-A453-1C010313D4B8@yahoo.com> To: bob prohaska X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-3.45 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.95)[-0.947]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.83:from]; BLOCKLISTDE_FAIL(0.00)[98.137.65.83:server fail]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.83:from] X-Rspamd-Queue-Id: 4QKCzy1Sdvz3l6X X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On May 14, 2023, at 12:31, Mark Millard wrote: > On May 14, 2023, at 08:36, bob prohaska wrote: >=20 >> Lately a Pi2 running -current has gotten stuck while buildworld is = running. >> There's no escape to debugger, no obvious errors on the console and = only >> modest swap use (tens of MB). So far the stoppages have been when = building >> clang. >>=20 >> One possible culprit is /boot/loader.conf, which has accumulated some = baggage >> over the years: >>=20 >> bob@www:/usr/src % more /boot/loader.conf >> # Configure USB OTG; see usb_template(4). >> hw.usb.template=3D3 >> umodem_load=3D"YES" >> # Disable the beastie menu and color >> beastie_disable=3D"YES" >> loader_color=3D"NO" >> vm.pageout_oom_seq=3D"4096" >> #vm.pfault_oom_attempts=3D"3" >> vm.pfault_oom_attempts=3D"120" >> vm.pfault_oom_wait=3D"20" >=20 > (I oroginally had a note here but I think it > would just confuse things and not be tied to > your problem.) . . . >=20 > However, I expect that your user process had > its kernel stack swapped out. See below. >=20 >> kern.cam.boot_delay=3D"20000" >> vfs.ffs.dotrimcons=3D"1" >> vfs.root_mount_always_wait=3D"1" >> filemon_load=3D"YES" >> net.inet.tcp.tolerate_missing_ts=3D"1" >> vm.swap_enabled=3D0 >> vm.swap_idle_enabled=3D0 >=20 > Those last two lines are for avoiding having > interactive sessions (sshd, serial console) > processes end up with their kernel stacks swapped > out. (But it does so by preventing such for all > kernel stacks, not just the ones of interest.) >=20 > When a kernel stack is swapped out for a > process/thread, the process/thread can not run > at all until the kernel stack is is read back > into the kernel. >=20 > Those last two lines you have in a file for > tunables --but are not tunables: >=20 > # sysctl -T vm.swap_enabled > # sysctl -T vm.swap_idle_enabled > #=20 >=20 > Compared that to the check for the writable > category: >=20 > # sysctl -W vm.swap_enabled > vm.swap_enabled: 0 > # sysctl -W vm.swap_idle_enabled > vm.swap_idle_enabled: 0 > #=20 >=20 > In my environment, I use /etc/sysctl.conf , which > is a place appropriate for non-tunable but writable > sysctl values: >=20 > # grep vm.swap_ /etc/sysctl.conf=20 > vm.swap_enabled=3D0 > vm.swap_idle_enabled=3D0 >=20 > I suggest moving the assignments to /etc/sysctl.conf . > I expect that this will get rid of your problem once > you reboot with them in a right place. (You can also > interactively set them via sysctl use.) >=20 > I suggest avoiding confusions by not having copies of > those 2 lines in /boot/loader.conf (where they will > not work). >=20 >> --More--(END) >>=20 >> However, the problem emerged well after the changes above were made. >>=20 >>=20 >> A running diary of experiments is at >> http://www.zefox.net/~fbsd/rpi2/crashes/20230514/armv7hang >=20 > There you report reducing the swap space partition size. > Were you getting the message about the swap possibly being > mistuned prior to that? >=20 > For 1 GiByte of RAM 3647M looks to me to likely be a little > below where that message about mistuning shows up. If you > were not getting the message, the size should have been > fine. >=20 > In other words, I expect it is appropriate to put back > the original size (or some approximation of it that > avoids the message about possibly being mistuned). >=20 >=20 > Everything that you reported looks to me to be consistent > with some kernel stacks having been swapped out for some > processes/threads that would otherwise be involved in > interactive I/O activity. >=20 Just adding an FYI about: load-time-Tunable and Writable vs. load-time-Tunable (spanning both writable and read-only ones) vs. Writable (spanning both tunable and not) and loader.conf vs. sysctl.conf appropriateness: # sysctl -aTW | wc # ( all: either loader.conf or sysctl.conf ) 431 860 12410 # sysctl -aT | wc # ( 617-431: just loader.conf ) 617 1231 17343 # sysctl -aW | wc # ( 1415-431: just sysctl.conf ) 1415 2821 42158 =3D=3D=3D Mark Millard marklmi at yahoo.com