From nobody Tue Nov 26 08:21:40 2024 X-Original-To: ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XyFsy2gT8z5f666 for ; Tue, 26 Nov 2024 08:21:58 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-8.consmr.mail.gq1.yahoo.com (sonic316-8.consmr.mail.gq1.yahoo.com [98.137.69.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XyFsx35bwz4cNH for ; Tue, 26 Nov 2024 08:21:57 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=qpOEkUSv; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.32 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732609314; bh=QIEZ2DNXMUtg+aiD7YkTdtaJBvLckFlUsUjdEF1Cg0A=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=qpOEkUSv6GupDlTbmV+Mao/Aj3au98HwkhML9EFtjm2wqde368JX6Rz6yEz14MBm+1l3aJDtsMz4aba73lgjfn/yMvzAsgki/9fNFii5o5AuQOQbL21XoTHwq8fc8ZeeajE1I1bI3VhMjytDhP66HVwgARvOcmGP4x1vS26F+EVaP/GVIrAkdSlfQg9/Z3jxhgaK/qn+IOsPW4uipkI80iRIC8VO8762I4S/7KryDySpQB29qNj8rzGAmOx/UascQDJsk+Uln18rAZX+KNd9j9halYfC6JE/B9dw2oE3Jz9WGc34QKt/obz5TQ8HD1SHZFGX5+hn+zkYia7lPktFoQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732609314; bh=v7wQoLy1sCxlomNBFGk1T9NSQm3r+d0L9LVxDYNgrEV=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=APF+WHmwwZesWOBC024O5FKnEQtNYwZVhlZUzAaN2dISonUwDun1E5v48x+Fw9UW+LBWUeXjd8LmQRfQsHhd6wM1GRynatLGGjczzmDCpd3GfmoRu5032B/Cv0bqwCXz8zBs9agDKmO1JBi7T9uz9LIEh5xGgO0jKeaMa2/5JSANXDWTCOxDDnkTPFccHilOgvN9EiTDTQeB0Cmw8o4bYeTvUl6ynOs5PLcw9x9NbjuxXXwqHOZvvwem7Ta2EI94a+xaU6xHN7Ac87TOml7PL8apsSHC7CBaepE9lqXg3XsUUnL6Spl+FNPP2J1gy7MMLU9OJWDdGx0uq+r/KRWtSA== X-YMail-OSG: _aotRj0VM1kexqeJGyK7lE0I2W0mpVvUbpWwdhYSeSnT1I1aYol_vf72mbj8Wgb 4j9ewMWqhh.jh2Ubevd0Oq.iWoOjsfgLxrZ0nh.1xNauGua2JCxu0H5205ml_zfZUBQpGYgMSe6C aGJ2nKQCfh1P64lLenpEj88.wU8HrRo4yhV3ggVEY72VkfSFAI.IZXDYdYA.2.TllDg.DqeVpZgO StXBL54YqjsesyaNryqAyN6N3C_fioCXHXPyevQSecLOloesOI6UyktcG7iDznd2zHRuKmUJ62MH lg4aXdYiqUjZeyux2VVpylLH2vzZXMoKWjUft3RXft8Fj3vJuiJsuWbqmDY1fWXBMu8u8q_HVSWf 8A4U1dfkpUWeIMOw6JoW4ewxGbOnAo2KuX0q9n7_VKxYJwaiI6MjbaV6DxML.2cI7lpAaUkG251e aYJmnDR07wYwevYsvGwxqy1YqtW9hpq1AUktyBVSE0EguSOeqBpW.ZNDK_oBoqZRZap7wI_o8FCA MHBFGtaLoiDvPkNbNvHLpI.pDLPdYfLxAmj28iR_.XZ9jZ2mMDMFJog.Vb_15WNb.voFiWRPvx5w Wc1J30MkRtILW7RA0ssu3lP3h0cFRe5wzP3K_CE87qPkdh2VKdRI9Ecn8daO_MBfSa2y.1Io9pDn bPt_KE5R.8K6EteKacDEgu3_p8jV79XkODMGGjoA0lcBQhnwpoG6oSqasXBnuBesAH._AV2n1QNV gSmEgROtrlaWHtuYvmP9_sFZVLdDjCtvdLc6GTbbh1mfbXwEJGPdVxKAqH5QDsEnJT93t9QeGurI 3yh62wDVRj1YEm222DT8kJa4Jvb9lz1QJgmdyDjJEPrGqzgShSj.ZOavsCQGUBh7h8Yes25lBUk7 1zdR.KIntXfUOfajYgIZQ6J87QLHq19jj3Um4rwrwOcNBiLmOc_rH2KCmX7K2_MKNxSCsZTDrnFG UXXWwC.EllgFX8VxiwY_U81ZegXicbgJ1KzGQoM9kJSHRWfryXIO6uxnIxLnIdh1uELQrPTfDr5X yOIX9Ng4miTc4KA7r.zrGsshHkr7DSk7W_jsOIaqQL7xNrrZlaoyfltB1wpW5Jsl.wcFDgV2pX6x AOCp0.2ezhyTsgg_2SEq.wIfwca60O6ZrryxlhN4MgPgQ_4neEv0s86SW.S.amK8UH.SNngqQaA. ELQaFKOBPKiPASaVmlfuu6BSdtipnCSkr7IlwMSqao1HJjhrOg8psLdOTX2APDIV2UKkhCvXcuPv ZU6xEaOCe_UJ0GKNa2495A_ZTrL2FZSJ9cUW9horE2CMb4qWBDYq43TizmcEmKX4_OdAGDDGo2.L sDHsTdINgekMErkBusuSeBsySYKf2rhq0QPN5aPJPiiA5NvkC_Id.dwfEKQ0XDw41GDTf8.7WKWa RGThiT9wRUfksn3Jt.G6opTlyZj7IpATjJDsZ1yDu79KObW_voCx15ay6I4T7jptAuKcmx5Hxlpg 6my0YhlFwjK4jHr_AFgHQHwrGQC4.yje0GF1cEm6x6A71W_mvqJV5LjWBworLHCgi4n9BGB7XDEj anXqbSv_Gcr1miID2ydhKLoFYoKsKgVKfUODt53PYsnZcW05OgxrDY3xZEsZ9fMReTLVNeCg.jGn pWvvgdp78MB0o0wNl4A1pg76bLR4G9Pt9cMYagsHEPjedS2OFkN5.R0ku2Oz8B3c6T7dIGMYRcKy tgeABGwjILdRKbssxzPgJfvua10IhwFtcWPKpPD78y3K00wEtkeAGzp9ph_KdcPB_I8fG9UOURBJ ssRlrCF7SZZVB9qd5VdG5bgX1m1ubVyf5FVgYI2NwX8WucVudf2o0ulmsPbwNngqT7RhWOMiyu.a q5Wg7tL_Vs7Rtv44bjNiTA50UEsuPFvBP6NArn.BaWkEnut1SCqSwfUyANtL1YMgVsCiG1kDSjZ3 atlAbhzTFpSAvLtqm2D7v1CO4aluccz1mZwQNLNoTDJ_uVJ7Nzp4KehdshEWIcSqYaUcdsgshfuL PnHXD5Nxdyh2oNftL8BqaOWaTmC9bH2m5YP8oelZ8mz_E6W2SzTm3SSRZ8wOUg3SvBTSU7e63Cul zHTHsS0BZg8H7KmyEPKxDREVzNmaa.Bnoo3gDI5Rj.eMbA1ND4C1nZSVqBHiXV3KB9TZMVHEJIP2 12rMe2MR58mUXyPP4_TIJnxHQZ6xmidFf5bT1UOhAxD6oNQbsCs5ldp_bdSOYXPh1EAknR0QTem1 iuMw31Zf3eagqY41GBa7UJ0us.VE9dBdXi4EbSjUtA.Vb33p9CXH37lEu.u8MU20Eli0qm6qlO0o KAsK7IN0VkD15XQYD_CbDsNe.0A-- X-Sonic-MF: X-Sonic-ID: ea04d37d-2b05-471e-8ebe-3e7371f0b010 Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Tue, 26 Nov 2024 08:21:54 +0000 Received: by hermes--production-gq1-5dd4b47f46-5qmz7 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 9288cee7416325367ace5b6e545e01a6; Tue, 26 Nov 2024 08:21:51 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-ports@freebsd.org Sender: owner-freebsd-ports@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] From: Mark Millard In-Reply-To: <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> Date: Tue, 26 Nov 2024 00:21:40 -0800 Cc: Dimitry Andric , Guido Falsi , =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , Yasuhiro Kimura , ports@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> References: <46E3A370-A3E0-4BAF-B707-87F94F98E248@FreeBSD.org> <5ee47c3d-f80e-4d50-9b6a-acb3c98e80e0@madpilot.net> <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> To: "jah@freebsd.org" , dougm@freebsd.org, asomers@freebsd.org, Mark Johnston , FreeBSD Current X-Mailer: Apple Mail (2.3776.700.51.11.1) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.995]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ARC_NA(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[ports@freebsd.org]; RCPT_COUNT_SEVEN(0.00)[10]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.32:from]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.32:from] X-Rspamd-Queue-Id: 4XyFsx35bwz4cNH X-Spamd-Bar: --- On Nov 25, 2024, at 22:10, Mark Millard wrote: > On Nov 25, 2024, at 18:05, Mark Millard wrote: >=20 >> Top posting going in a different direction that >> established a way to control the behavior in my >> context . . . >=20 > For folks new to the discoveries: the context here > is poudriere bulk builds, for USE_TMPFS=3Dall vs. > USE_TMPFS=3Dno . My test context is amd64 on a > 7950X3D system with 192 GiBytes of RAM. Others have > other contexts, including an Intel system. >=20 >> I changed USE_TMPFS=3Dall to USE_TMPFS=3Dno : >>=20 >> USE_TMPFS=3Dall gets the failure >=20 > Note: The test case is corruptions of the likes of parts of > the .got.plt in libsass.so.1.0.0 from text/proc/libsass . > The corruptions are well 4 KiByte aligned blocks of zeros > showing up in the files that should not be that way. >=20 > 2 examples of bad libsass.so.1.0.0 builds have: >=20 > Contents of section .got.plt: > 2bed60 00000000 00000000 00000000 00000000 ................ > . . . > 2befc0 00000000 00000000 00000000 00000000 ................ > 2befd0 00000000 00000000 00000000 00000000 ................ > 2befe0 00000000 00000000 00000000 00000000 ................ > 2beff0 00000000 00000000 00000000 00000000 ................ > 2bf000 96ab2a00 00000000 a6ab2a00 00000000 ..*.......*..... > 2bf010 b6ab2a00 00000000 c6ab2a00 00000000 ..*.......*..... > 2bf020 d6ab2a00 00000000 e6ab2a00 00000000 ..*.......*..... > 2bf030 f6ab2a00 00000000 06ac2a00 00000000 ..*.......*..... > . . . >=20 > Contents of section .got.plt: > 2bed60 00000000 00000000 00000000 00000000 ................ > . . . > 2befc0 00000000 00000000 00000000 00000000 ................ > 2befd0 00000000 00000000 00000000 00000000 ................ > 2befe0 00000000 00000000 00000000 00000000 ................ > 2beff0 00000000 00000000 00000000 00000000 ................ > 2bf000 00000000 00000000 00000000 00000000 ................ > 2bf010 00000000 00000000 00000000 00000000 ................ > 2bf020 00000000 00000000 00000000 00000000 ................ > 2bf030 00000000 00000000 00000000 00000000 ................ > . . . > 2bffc0 00000000 00000000 00000000 00000000 ................ > 2bffd0 00000000 00000000 00000000 00000000 ................ > 2bffe0 00000000 00000000 00000000 00000000 ................ > 2bfff0 00000000 00000000 00000000 00000000 ................ > 2c0000 96cb2a00 00000000 a6cb2a00 00000000 ..*.......*..... > 2c0010 b6cb2a00 00000000 c6cb2a00 00000000 ..*.......*..... > 2c0020 d6cb2a00 00000000 e6cb2a00 00000000 ..*.......*..... > 2c0030 f6cb2a00 00000000 06cc2a00 00000000 ..*.......*..... > . . . >=20 > So: Where the zeros end varies but the start of > good data end up's at some 0x...000 offset: a > multiple of 4 KiBytes. >=20 >> vs. >> USE_TMPFS=3Dno works just fine >>=20 >> So it is a FreeBSD system error associated with >> use of tmpfs . >=20 > Recent work on tmpfs includes: >=20 > Mon, 09 Sep 2024 > =E2=80=A2 git: 8fa5e0f21fd1 - main - tmpfs: Account for whiteouts = during rename/rmdir Jason A. Harmening > Fri, 04 Oct 2024 > =E2=80=A2 git: 75734c4360fc - main - tmpfs: check residence in = data_locked Doug Moore > Sun, 13 Oct 2024 > =E2=80=A2 git: ec22e705c266 - main - tmpfs: remove duplicate flags = check in tmpfs_rmdir Alan Somers > Thu, 24 Oct 2024 > =E2=80=A2 git: db08b0b04dec - main - tmpfs_vnops: move swap work to = swap_pager Doug Moore >=20 > swap_pager (given the reference to it above): >=20 > Tue, 08 Oct 2024 > =E2=80=A2 git: d0b225d16418 - main - swap_pager: use iterators in = swp_pager_meta_build Doug Moore > Fri, 11 Oct 2024 > =E2=80=A2 git: 1107834090be - main - swap_pager: swapoff detecting = object death Doug Moore > Thu, 24 Oct 2024 > =E2=80=A2 git: 34951b0b9e78 - main - swap_pager: move = scan_all_shadowed, use iterators Doug Moore > =E2=80=A2 git: 02e85d1c8a41 - main - swap_pager: fix assert in = seek_data Doug Moore=20 > =E2=80=A2 git: faa9356f97d2 - main - swap_pager: fix seek_hole = assert Doug Moore > Sat, 26 Oct 2024 > =E2=80=A2 git: 39f6d1e7f835 - main - swap_pager: iter in haspage, = lookup, getpages Doug Moore > Wed, 13 Nov 2024 > =E2=80=A2 git: d11d407aee48 - main - swap_pager: Ensure that = swapoff puts swapped-in pages in page queues Mark Johnston >=20 > I do not know at this time when the corruptions started. The > above is only suggestive. With a bulk -i active but from outside the bulk -i : # df -m | sort -k6,6 | grep ^tmpfs tmpfs = 182907 0 182907 0% = /usr/local/poudriere/data/.m/main-amd64-default tmpfs = 184770 1863 182907 1% = /usr/local/poudriere/data/.m/main-amd64-default/ref tmpfs = 2048 45 2002 2% = /usr/local/poudriere/data/.m/main-amd64-default/ref/.p tmpfs = 182907 0 182907 0% = /usr/local/poudriere/data/.m/main-amd64-default/ref/var/db/ports Note: bulk -i lands one in = /usr/local/poudriere/data/.m/main-amd64-default/ref/ =46rom inside a bulk -i where I did a manual make command after it built and installed libsass.so.1.0.0 . The manual make produced a /wrkdirs/ : # find -s / -name libsass.so.1.0.0 -exec ls -ilodT {} \; 6417 -rwxr-xr-x 1 root wheel - 42444424 Nov 26 07:24:37 2024 = /usr/local/lib/libsass.so.1.0.0 11872 -rwxr-xr-x 1 root wheel - 42444424 Nov 26 07:26:48 2024 = /wrkdirs/usr/ports/textproc/libsass/work/libsass-3.6.6/src/.libs/libsass.s= o.1.0.0 12294 -rwxr-xr-x 1 root wheel - 42444424 Nov 26 07:26:48 2024 = /wrkdirs/usr/ports/textproc/libsass/work/stage/usr/local/lib/libsass.so.1.= 0.0 # objdump -hs = /wrkdirs/usr/ports/textproc/libsass/work/libsass-3.6.6/src/.libs/libsass.s= o.1.0.0 | less . . . 2bed60 78ba2b00 00000000 00000000 00000000 x.+............. 2bed70 00000000 00000000 86a62a00 00000000 ..........*..... 2bed80 96a62a00 00000000 a6a62a00 00000000 ..*.......*..... 2bed90 b6a62a00 00000000 c6a62a00 00000000 ..*.......*..... . . . So the original creation looks okay. But . . . # objdump -hs = /wrkdirs/usr/ports/textproc/libsass/work/stage/usr/local/lib/libsass.so.1.= 0.0 | less . . . 2bed60 00000000 00000000 00000000 00000000 ................ 2bed70 00000000 00000000 00000000 00000000 ................ 2bed80 00000000 00000000 00000000 00000000 ................ 2bed90 00000000 00000000 00000000 00000000 ................ . . . 2befc0 00000000 00000000 00000000 00000000 ................ 2befd0 00000000 00000000 00000000 00000000 ................ 2befe0 00000000 00000000 00000000 00000000 ................ 2beff0 00000000 00000000 00000000 00000000 ................ 2bf000 96ab2a00 00000000 a6ab2a00 00000000 ..*.......*..... 2bf010 b6ab2a00 00000000 c6ab2a00 00000000 ..*.......*..... 2bf020 d6ab2a00 00000000 e6ab2a00 00000000 ..*.......*..... 2bf030 f6ab2a00 00000000 06ac2a00 00000000 ..*.......*..... . . . So: The later, staged copy is a bad copy. Both are in the tmpfs. So copying to the staging area makes a corrupted copy inside the same tmpfs. After that, further copies of staging's bad copy can be expected to be messed up. =3D=3D=3D Mark Millard marklmi at yahoo.com