From nobody Thu Nov 28 20:29:09 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XznwN65Blz5fDhF for ; Thu, 28 Nov 2024 20:29:24 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-25.consmr.mail.gq1.yahoo.com (sonic304-25.consmr.mail.gq1.yahoo.com [98.137.68.206]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XznwM2X1mz4ryN for ; Thu, 28 Nov 2024 20:29:23 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b="WT/rHaK1"; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.206 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732825761; bh=BzM/7JCOnR7iAW1KItZB4zzZ74yUhH29Je1jnF6OG3c=; h=From:Subject:Date:Cc:To:References:From:Subject:Reply-To; b=WT/rHaK1XHDD8O3k3h+zKqD3I7aE3XLT3HWUiU5X2Z8OYYxeCDQm5xkxKARUhDKL8NSMqcp0U/MNwH8qpRDr4fg5VGSAAN8OtggrYbZhgxKl2L0ZaWVUtjB7b39TCcrtl8JnSP03rM4VdiQ4vGQhTdy8w2NU5rra0z6vWZPWJFySVzTx+Kn2oEz5O2oroTpHyFnZ8m5K9T1eEDdfDUBEZhrQsKAGPEaFjOTKEnbblvgT9UeQoxr6kVFZvkP7sTGaDhelx4Z5a+EQpVdbcIepYkcKV0c+uir7Xo559Bu3VwK6+oe6uSXpeLagC1YTlpOTe0OybtwYjGfSlQnfDj4FqQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732825761; bh=uhMNetRXk6WGxyAixx3PW+Ukb2rhD6G9NUSt18v34W9=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=Z3EzVxvB/gD1Ho6HxGizUmmfBQPRkX4bUsq+WpyYqNYoe5rDaOPihTC5Wa4V4EwQVreG32Wn8qD+BcGDI5TCrMv/Q3hyKa3OQahXgMgVuw1W0LsHicTte5TbIo3QJ43SZzN6SB6iuFnSZ6RQOcflnXxk8Wn5oI6bYODziFyymdPKQbTKgeRLEueXeR/zEAwvE3bL69aRQCP6bhNY3PMnzbO3kAtEci78F0QV3rrNxm0Liu6PO9HyXYwYTN7vfSL4X2r93OkyJm9gooHLGI4glp8Y+2Fhd+y/aAkp/1LprtUazmdeNYSu00hfcHOY/ZKGECpVZy+NvcQpJBKLlRWycw== X-YMail-OSG: 0vEZIgYVM1nM.GYTpQpiymv2zlIkc49iYmh6TKC4Q6Zsufup5t3Yv9gmJulPnUa ST7sQUQueukQt3eVwcEPoQyjOJsb2WkFnvi0wGQpVzQt74kpq5meB4dAzp73aThoGlkAG9cEdnAv SNcyOcKyByW0BqVuCOrC5D0Uf2XRxzg99x02mjxHOCv5GAPCrYHzRWptQyQ8AOYYcBjpjvEuE2zS 5uPMJw2IjZBhZMQ8F2YPUgpiP0jyn5SX2cz1791zrVS7501YzNcMykvstnmP4_LTMa834TdA0NMG CUFdqyoec0L4TeIVXrHz_W.mY0BcAhNTlExkgewEFiNd0f1Bpd9NqzO3Mx.fmru_BdCRwaE1obwP WXJvP41VtFCRRP0YQTGjo5BzwdxjzETZHAyxcwWNeA23Xz77_70ELYHSazI0cT2AEqJqpqr8RhgR VoFI_A8JXgs7LZO8.0kw5GztzgKpY2JZBG_nwnbwPA7wzdOjBnMrRC2U2jWAjgdKQrOm6O27WADL 3RjlAD1wLwd5raX8.K6y.eFMJz89f2tJog0hAem_WMiMZPutmMDFmcAJ422oEt4RfKpLNzM9o9XN T6dj.WF_4qnAXHceOv5TBm.75uusnMUHs4btfHP9EV_Vfap7hqltQvFxYTa9UZdlpsg.DXfDnNk. buIvroDqfUrKnQh3_vpetET69Um0TeeVug8XkKNMfxnScv9Fy63iuINDFQbjHA1aQ3drfhvYeu9M 7bB.7ROfbzu4czrQk3MQZFcz06sOHJ.dQF31VjMjnlVIhCytZU_SSceuujjh6q1sNf2NQqrh6vuI IJ2BmL4NOOKetOFkSKn6tHp4txW5xBcKyxbuPlnd7qUjRfzkZq1WVGNCaLoU0PBO9Z9zy23px1R8 H91u3MaXyXAdgXtbJruT6o9nComhbtaefvjGLEbguQ2tev9IK7P.R_JjnR6Ht1gFucx7xOOEpYa0 ds85wqjSAhc9WX6bOpGjEmeDzivbimqWPa0jYVYSWEtPd6uCJ_rGVoq1U8DQKypZ_hyr0okP6f.3 ynJuoDgpLeeX6HjlWyvfzZI_QZO7op2fx1X3_WsXcbOVNIi.h5yyIX0m.49OJCWfYF3Ug3sZ7fzQ Bzj7rn3b9dch7o9tMyqWIchcJFflUOU.4IazDOeEv4pIsP9McEJYTnkwSwiV9IVC42hM_3Uunkj8 OxneeTx4qJXGjEc3Tk2qccJrlOjbuQgwR.ug0naeao2f6g22p4cilgG_uFX8rOQD5GQIE9tXElC9 z4.y6cYrZERYbxYpg3qPmkFcD5iuAz3MqprXufq_YMC4fI9pss6AEpzP2W3ulv07pfGzpZCsfn8F 77vbXhqF_9SpfbE_d7sgjDAuthY3Uh0m_H_FXXgCTdIVrK._PV5LAtW.d7UpAjywETk8zahnpHKt 3Bh2BQBfRI54Kzjg3RKxhR2XqCkSGITlc.6TUJFllG3OaVm4HHYeiYZY6GDWFNGNZEWUH5org_q4 X9ukid4uQShjzJG8ihEwJRikjmWumKJj3n9f95efs4jdKieQJObB9.ZveUqwwYCsAZzKLcv0Aui2 Shm6.p431ueIGNpCdx9McxLWGOZZw4kZ.Xenm88Z0cfEvMWZhClPcY2kn0LJTLPLWNw2.wLxUIXQ YgVLevaPc2u1eOllmzhatwyxdkBVEG8XobIySbjop9Y4ZyJiu6sQq5661SzFI1lS4.ftfJYtrq5e gyUvmD4dxfgNx9bHcZJQqk8u01DS0ITumeCfAkWzmUQVv.zBFhTLA_ksAL81B.Y8cC7FmBY36r8c ZKC7dH624i.h3XnvpM76FamzCU6f9oSs3r38fV0JpCRiCOy1jhY4x8No4EOarDv2ee5A.ZqopGRh XCo_DhiETOJYurT._UI9UXA.dyxfRgTkkrB2CXlvnUF44twT1ri0iyB6RMyvhd11CF0p3YZCIoyR zIu6s8tEi0.8mtHjlKiKROWV3O_jqRxOaMKHPZmIgM4ZKbOt_h55PNiKb_ANLx2qec0p_BZHy6VY K746bEYduGe0KRPZe3358jVtsBUgpyJxHVNd4Few83cGEfdsQMw1XvNOxJPltyVH0KzWJhTr53Gk 9HQDnsZoO4klD5WDWFZUt2AeAXiK.DtVsI4m4_6rbqJk27DjNGAjUJLfEjsyjTpZPhlXBsut68iU t8bf.lGeSBaCiQwz8vo48GD0R2Lk6e3U6J8m5pAlUmKv_GtgAjHRK9c8WR5B7djoYz3ud0giVXTs WaVPeK3l3.I3IbvE2thP4rUO_IPtEU3RSKQl8kIxT49vyfH1ehdG2yIzF.jCAlA9y X-Sonic-MF: X-Sonic-ID: 43578921-9275-41c9-8520-a7661edc6142 Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Thu, 28 Nov 2024 20:29:21 +0000 Received: by hermes--production-gq1-5dd4b47f46-fhdpd (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 661dd44bf463b81a4e885253030570c6; Thu, 28 Nov 2024 20:29:20 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] Message-Id: <4D541C32-7DF4-4CAC-B31E-D4DD17977154@yahoo.com> Date: Thu, 28 Nov 2024 12:29:09 -0800 Cc: FreeBSD Current , FreeBSD Mailing List To: "scf@freebsd.org" X-Mailer: Apple Mail (2.3776.700.51.11.1) References: <4D541C32-7DF4-4CAC-B31E-D4DD17977154.ref@yahoo.com> X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FREEMAIL_FROM(0.00)[yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.206:from]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.206:from] X-Rspamd-Queue-Id: 4XznwM2X1mz4ryN X-Spamd-Bar: --- Sean C. Farley wrote on Date: Thu, 28 Nov 2024 18:16:16 UTC : > On Mon, 25 Nov 2024, Mark Millard wrote: >=20 > > On Nov 25, 2024, at 18:05, Mark Millard wrote: > > > >> Top posting going in a different direction that > >> established a way to control the behavior in my > >> context . . . > > > > For folks new to the discoveries: the context here > > is poudriere bulk builds, for USE_TMPFS=3Dall vs. > > USE_TMPFS=3Dno . My test context is amd64 on a > > 7950X3D system with 192 GiBytes of RAM. Others have > > other contexts, including an Intel system. >=20 > I have been seeing some odd behavior from Firefox as well as with=20 > poudriere builds on my system. Both of which are touching a tmpfs=20 > system as I have setup /tmp as tmpfs, which Firefox uses, and=20 > USE_TMPFS=3Dall. >=20 > The system has been an experiment, for me, with undervolting. I have=20= > been attributing any flakiness to the undervolting, but I have reduced=20= > that a lot while the instability has been consistent as in it has = stayed=20 > rare. I cannot tell how many times I have run memtest86 on this = system. >=20 > System setup: > - FreeBSD 14.2-STABLE The context that I investigated --and what was fixed by a commit only applies to-- main [so; 15 as stands], not stable/14 . stable/14 has no commits mentioning "tmpfs" after 2024-Jun-04. > - i7-14700K (latest BIOS which *should* fix Intel power-related bugs) > - 128 GiB RAM > - ZFS (mirrored drives) The primary test context was ZFS but no redundancy or such. (Only really used for bectl activity.) But testing on a UFS copy of the live directory tree also got the problem. The actual problem was in tmpfs support. > - 2 encrypted swap partitions (64 GiB each, lightly used) No encryption involved in my context at all. > - Lightly undervolted (-0.06 offset to Global Core SVID Voltage) Nothing analogous in my context. > - /tmp is tmpfs I have no default areas that are tmpfs: so only what poudriere temporarily created during the bulk builds. > - ${HOME}/.cache is tmpfs No use of ccache or the like. > - Poudriere: > - USE_TMPFS=3Dall I also use TMPFS_BLACKLIST . My personal environment causes use of -gline-tables-only as debug information normally. (That option is clang/clang++ specific. gcc* and clang* do not seem to have a common notation for analogous settings on the command line.) > - ccache No use of ccache or the like. > - jail version in sync with host True for my context. But the issue that was fixed was in the kernel code, not the world code. > - /usr/ports is mounted with nullfs Also true for my context. > I have wondered if it was swap-related, but recently I noticed a build=20= > failure with games/veloren-weekly where swap was available but zero=20 > bytes were used. The system was under little load at the time so less=20= > chance of undervolting being an issue. >=20 > Build failure: > ----------------------------- >=20 > portpicker =3D { path =3D = '/wrkdirs/usr/ports/games/veloren-weekly/work/portpicker-rs-df6b37872f3586= ac3b21d08b56c8ec7cd92fb172' } > =3D=3D=3D> Updating Cargo.lock > error: checksum for `windows_x86_64_msvc v0.42.2` changed between lock = files >=20 > this could be indicative of a few possible errors: >=20 > * the lock file is corrupt > * a replacement source in use (e.g., a mirror) returned a different = checksum > * the source itself may be corrupt in one way or another >=20 > unable to verify that `windows_x86_64_msvc v0.42.2` is the same as = when the lockfile was generated >=20 > *** Error code 101 >=20 > ----------------------------- >=20 > Restarting the build finished successfully. >=20 > >> I changed USE_TMPFS=3Dall to USE_TMPFS=3Dno : > >> > >> USE_TMPFS=3Dall gets the failure >=20 > *snip* >=20 > >> vs. > >> USE_TMPFS=3Dno works just fine > >> > >> So it is a FreeBSD system error associated with > >> use of tmpfs . > > > > Recent work on tmpfs includes: None of this is directly stable/14 : all main [so: 15 as stands]. stable/14 has no commits mentioning "tmpfs" after 2024-Jun-04. So none of these changes are involved for stable/14 . > > > > Mon, 09 Sep 2024 > > =C3=A2=E2=82=AC=C2=A2 git: 8fa5e0f21fd1 - main - tmpfs: Account for = whiteouts during rename/rmdir Jason A. Harmening > > Fri, 04 Oct 2024 > > =C3=A2=E2=82=AC=C2=A2 git: 75734c4360fc - main - tmpfs: check = residence in data_locked Doug Moore > > Sun, 13 Oct 2024 > > =C3=A2=E2=82=AC=C2=A2 git: ec22e705c266 - main - tmpfs: remove = duplicate flags check in tmpfs_rmdir Alan Somers > > Thu, 24 Oct 2024 > > =C3=A2=E2=82=AC=C2=A2 git: db08b0b04dec - main - tmpfs_vnops: move = swap work to swap_pager Doug Moore > > > > swap_pager (given the reference to it above): > > > > Tue, 08 Oct 2024 > > =C3=A2=E2=82=AC=C2=A2 git: d0b225d16418 - main - swap_pager: use = iterators in swp_pager_meta_build Doug Moore > > Fri, 11 Oct 2024 > > =C3=A2=E2=82=AC=C2=A2 git: 1107834090be - main - swap_pager: swapoff = detecting object death Doug Moore > > Thu, 24 Oct 2024 > > =C3=A2=E2=82=AC=C2=A2 git: 34951b0b9e78 - main - swap_pager: move = scan_all_shadowed, use iterators Doug Moore > > =C3=A2=E2=82=AC=C2=A2 git: 02e85d1c8a41 - main - swap_pager: fix = assert in seek_data Doug Moore > > =C3=A2=E2=82=AC=C2=A2 git: faa9356f97d2 - main - swap_pager: fix = seek_hole assert Doug Moore > > Sat, 26 Oct 2024 > > =C3=A2=E2=82=AC=C2=A2 git: 39f6d1e7f835 - main - swap_pager: iter in = haspage, lookup, getpages Doug Moore > > Wed, 13 Nov 2024 > > =C3=A2=E2=82=AC=C2=A2 git: d11d407aee48 - main - swap_pager: Ensure = that swapoff puts swapped-in pages in page queues Mark Johnston > > > > I do not know at this time when the corruptions started. The > > above is only suggestive. >=20 > Thank you for listing those. >=20 > I need to find some time to look over those changes although I am no=20= > kernel guru by a long shot. However, I see now that it looks like much=20= > more knowledgeable people are already looking on the current mailing=20= > list at the issue. None of them were applied to stable/14 . =3D=3D=3D Mark Millard marklmi at yahoo.com