From nobody Thu Nov 18 20:15:24 2021 X-Original-To: arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E4AD0188C49B for ; Thu, 18 Nov 2021 20:15:35 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-20.consmr.mail.gq1.yahoo.com (sonic306-20.consmr.mail.gq1.yahoo.com [98.137.68.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hw9yt3206z4gls for ; Thu, 18 Nov 2021 20:15:34 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1637266527; bh=J5jTDIaTnEZkZoigEBqXE+AgCkZprZHN4kz4vMa+nd0=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=jPQ+PN9rOwtoNQdj8UK47+zCMS3Xq+pIIIQJzafw4K+cVI0pR0CqAc7gGXMDEvp5VYsU+D2UFkayiiMmWt+G+LdBmU8EkzqNGsN+9GJTBrmkL52mTUvQmQKHTx8sMRYkKNVaimclqqr1hYxN2v2YA3Reh4pKplmlH0iq9iaB7qI/4X8drFaQv31f5dFguj/oBnTP87oG3XZy+RSQYSqw8zGm0gOP6RXEqJiU1+N0iPPZGi6E9yjsubjgfn9RQJYX2apo33fO8tj/vdFQtyWpgXNbEOZtSofuFXKaJX5Ea3F/DYCj3gsz5pu1K2l5Rpn7KrGZ+o10ExgaHsih7oig+g== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1637266527; bh=22TNxPffxf7O4ZS25ulyT9IpWTp0oSxF8C2IRXdmggt=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=OtPgkFlsYOZo0ueOEput32mGu4zEz0lY1jhF00IOYf96dHozTDt+aOzrbW4paCu3qy10YDheDpM+awhcmpZMA4BpAdBQ+DPplpaUok5tDF5kIrW1ZhbGqwLXWgmHHv3ckNI/UQ+ItEs7aSdE5CeS0dMW35+F8O5wum/XGvS/jsfAIC4k1TSOna+RP0Ke+O6yyQ1ncgTkmF/ALLp/z9TULlD3hQ/SFNd9O3k15JtPpMdsfoCbsOr6ow8ur65eRWPJK6Ltdn2r3aAuU/62cvIjUuB0Xdft8vF9TFuEX86Dw4U0qs9gidY/4GGiMmaiTV27jQ+x+R4b0bT0v4xgFZoNNQ== X-YMail-OSG: iJQVXacVM1lNwdySzIF0T1h4BvJBeaVlYFk7da7Wlyef06G2QJsNQPWp4YmSNX_ CZsWrJYYV73UWrvCzb9B8zbrNNjAgXpI45qklwTp5LMgnwJnc9ho32DKcYH79p8Yz7ksPNwFaPpk Dg9NKKoPGL0XjhLWf6zLA.okwH4h1cwzYglZ3GoSkArxGPJ.E2nm8Unn5epmBzU08zfN3Crt_z2a VXxaJ9DWorh1NqCgfAvYqLFumnKzS9E3gYsyY2ioT7kWKAbDdlgBrIcZG18vqMPIvx.HdVB3B033 1srElwg_9Jqp0V.ksFYgg6FMh.a2FB.cxq5lyByc3ikDNAMl3bxq3ZwHulnZNb_iXq2hJiLyqrXH m65lXZ_ts6knfXPwdNqQzqY6dmCZ.iqD5cka7rkc8e3Ld5SuLRrVRUDR8kVwY24h3W7iouculGmR Cq_pNws01zBEp0WkPEzu5oPsGrl5KlqUE1gLbVbNKtCTUoC05735.5dQuHkVh1KblyBtUw9cQrwf YJCFZoNuV9haSeEaWwFb5MrL5s7Y.2l.5fX6_TKq_Qq5XayK7oh.wXMxoycJTHlr0WgHQG8Ax._W nAIs_8cHrSZWL3xIOf2c0Y9CfxbyLo0TFnLNXAItShL1DfNzwuWfH7KK7rWL_OS7Nsap7CO7YVkE XLHHAiz8PF0pVaIYknGNb4Ml67Pn.loYpsrMe51ON_6KhBIQN79m9I8kjYF0AI.ve31JVenWEo9v UDnot1sYmASroxfvjwlnpX3VIvsmL1gmvq4WLPcpSGo7FEp.HPjZmJJBQ5_9Bc0wSu81mPZKbtOq s52b5NVTija.hMVmyMPUIlGxawoSTXFThc5n6BWmCB0g6VPyxwzJIGZXagnqtCvbRt3kbXAIgCIQ fW7v7leydEDEIyf.eX7FrJ6mrsM6JDXOld409Gf3hE8cR7.4HX_D9hVrc_7okfLoku4qLrYwADZB Wqn_pDjVwbHsKjX1.J1M3_NLts4qDdHbcfZIFYRXh4Mu6xT7Wd4fg0KcFt9lKaOm5VK7y4AdQiDv mzwYE1U50lajNorcvRUbmX91rtvFeXD3snrgHJZ7Xc.NZBe2fYWs2G0ar1Zo8LDBuh5aYr56N37H c26QLoDveSAmcKr6hek1WUcUqTrPZlzgfuN9wDfxHQozmw4BRw5eiCAXObHKcEctagoS2z9_KnQ9 yLs8oaqNoK8Zx40866_CefYGOVP3jXddPt5TB5smy7yO2ScNNenfpQX.rPZRTQmLJduwt9Yx5nEI ce0Y_uEZFjFJ3XtZDAS3aYPB_TWRxHn1PlpbZdNvYrfsnkgzG2XqU0wsE1DYp8dHWfwZ9LdyudOZ OPLq7AbQdB5PTZIL5kONxfqe_whxBRgH3n8_eyL3J44zhOpJJgc6vQH3MEFxBW6wUMslAwfreES_ RD6fYvXwUJ5dLBl9btC1aszAMilSJIxNnTSey8YuQ.wgRcS_iN46UNUZj793i0pGwhkUvl9WCasy ieShb__19pguVvZ7QwiG4eZhA_T0ThkwjpDo_3sGVPSEJfhWBDzDYHKRgLrg9O_YWakpHNTBmusc dc4j_oi7xWl.glPbGtLv8y33aRvMXNHdx_4Hs54NIboo2kWYg4qzsXEmiycWTDG8yhXDZFSKVuF2 eRHdHL7CYaW_QNeNRthGRUWVHBEuduMa2UNThOVT2xDYecmQZlXa5PDx3TLK.wZTYWd0UVHXqSO5 U3hcFDPUfJ69eb2iNrGmDVjie3ggejxzEG_R2.TaN92P2If143nLEfypUPvO4.JU8SSLlyHOKiNl uTwXbOMUVgWERuMxFzAie4KSHM5eoYhYIYZub6oCqJ2F_SHW59smPb9daoOuk0BjF0vop0CV7ry9 DLSVb_6eIfdATNiCX_kdEK5zO5qHmFWr6WpBeeE8x2LuN_99pRQS76_zfcXI4_Kg_jCjzZais2E3 IDIF8wlg8YlriN3CBSsu4Y8Pi1k5OVVd2DKbNwwWz9QBKoNP9mSrDmNSfHPY8YYP762sQnpHvR53 CSATrluBi.bneRbwX97o7aIMUobVMWHEuISSDKlmDTeEoDcX.XkPhCP_JvcdFhQSu5U.Y8UU9iYh RJj46enm.juku.UMc_3bvgk82NT39UKd5bv0HOnhcN6dnLppnM.mxyfTsmv2tNiQos0C4Z2EZ7yv LFjAxMlRxuY3D1ytbyj_TN4whiSo3aqnDjOci7VgD5gI5LBDOaSO6UQ.u10MumC1LMQk.rOs1wji VTlJeObJpjsPYm_XjrRU7._c3PczcRfXskLuU6Qff.HfRNA-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Thu, 18 Nov 2021 20:15:27 +0000 Received: by kubenode524.mail-prod1.omega.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 587cd65cf2b323b8c51e3fe72ea561ae; Thu, 18 Nov 2021 20:15:25 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: aarch64(?) poudiere-devel based builds seem to get fairly-rare corrupted files after recent system update(s?) Date: Thu, 18 Nov 2021 12:15:24 -0800 References: <2CA61249-321C-45AA-9755-597146AB8E9F@yahoo.com> <65AA4BCD-EC4B-4A19-B750-C7FC6E5ADDF5@yahoo.com> <9BF4F65B-6437-4D88-AF34-9BCFBF90D6F3@yahoo.com> To: freebsd-current , "freebsd-arm@freebsd.org" In-Reply-To: Message-Id: <4B591638-4693-4403-8549-88D7A1D9D669@yahoo.com> X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Rspamd-Queue-Id: 4Hw9yt3206z4gls X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=jPQ+PN9r; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.83 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-3.50 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.83:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.83:from]; RCVD_COUNT_TWO(0.00)[2] Reply-To: marklmi@yahoo.com From: Mark Millard via freebsd-current X-Original-From: Mark Millard X-ThisMailContainsUnwantedMimeParts: N On 2021-Nov-17, at 11:17, Mark Millard wrote: > On 2021-Nov-15, at 15:43, Mark Millard wrote: >=20 >> On 2021-Nov-15, at 13:13, Mark Millard wrote: >>=20 >>> On 2021-Nov-15, at 12:51, Mark Millard wrote: >>>=20 >>>> On 2021-Nov-15, at 11:31, Mark Millard wrote: >>>>=20 >>>>> I updated from (shown a system that I've not updated yet): >>>>>=20 >>>>> # uname -apKU >>>>> FreeBSD CA72_4c8G_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #18 = main-n250455-890cae197737-dirty: Thu Nov 4 13:43:17 PDT 2021 = root@CA72_16Gp_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm6= 4.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64=20 >>>>> 1400040 1400040 >>>>>=20 >>>>> to: >>>>>=20 >>>>> # uname -apKU >>>>> FreeBSD CA72_16Gp_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #19 = main-n250667-20aa359773be-dirty: Sun Nov 14 02:57:32 PST 2021 = root@CA72_16Gp_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm6= 4.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1400042 1400042 >>>>>=20 >>>>> and then updated /usr/ports/ and started poudriere-devel based = builds of >>>>> the ports I's set up to use. However my last round of port builds = from >>>>> a general update of /usr/ports/ were on 2021-10-23 before either = of the >>>>> above. >>>>>=20 >>>>> I've had at least two files that seem to be corrupted, where a = later part >>>>> of the build hits problematical file(s) from earlier build = activity. For >>>>> example: >>>>>=20 >>>>> /usr/local/include/X11/extensions/XvMC.h:1:1: warning: null = character ignored [-Wnull-character] >>>>> =20 >>>>> ^ >>>>> /usr/local/include/X11/extensions/XvMC.h:1:2: warning: null = character ignored [-Wnull-character] >>>>> >>>>> ^ >>>>> /usr/local/include/X11/extensions/XvMC.h:1:3: warning: null = character ignored [-Wnull-character] >>>>> =20 >>>>> ^ =20 >>>>> /usr/local/include/X11/extensions/XvMC.h:1:4: warning: null = character ignored [-Wnull-character] >>>>> >>>>> ^ >>>>> . . . >>>>>=20 >>>>> Removing the xorgproto-2021.4 package and rebuilding via >>>>> poudiere-devel did not get a failure of any ports dependent >>>>> on it. >>>>>=20 >>>>> This was from a use of: >>>>>=20 >>>>> # poudriere jail -j13_0R-CA7 -i >>>>> Jail name: 13_0R-CA7 >>>>> Jail version: 13.0-RELEASE-p5 >>>>> Jail arch: arm.armv7 >>>>> Jail method: null >>>>> Jail mount: /usr/obj/DESTDIRs/13_0R-CA7-poud >>>>> Jail fs: =20 >>>>> Jail updated: 2021-11-04 01:48:49 >>>>> Jail pkgbase: disabled >>>>>=20 >>>>> but another not-investigated example was from: >>>>>=20 >>>>> # poudriere jail -j13_0R-CA72 -i >>>>> Jail name: 13_0R-CA72 >>>>> Jail version: 13.0-RELEASE-p5 >>>>> Jail arch: arm64.aarch64 >>>>> Jail method: null >>>>> Jail mount: /usr/obj/DESTDIRs/13_0R-CA72-poud >>>>> Jail fs: =20 >>>>> Jail updated: 2021-11-04 01:48:01 >>>>> Jail pkgbase: disabled >>>>>=20 >>>>> (so no 32-bit COMPAT involved). The apparent corruption >>>>> was in a different port (autoconfig, noticed by the >>>>> build of automake failing via config reporting >>>>> /usr/local/share/autoconf-2.69/autoconf/autoconf.m4f >>>>> being rejected). >>>>>=20 >>>>> /usr/obj/DESTDIRs/13_0R-CA7-poud/ and >>>>> /usr/obj/DESTDIRs/13_0R-CA72-poud/ and the like track the >>>>> system versions. >>>>>=20 >>>>> The media is an Optane 960 in the PCIe slot of a HoneyComb >>>>> (16 Cortex-A72's). The context is a root on ZFS one, ZFS >>>>> used in order to have bectl, not redundancy. >>>>>=20 >>>>> The ThreadRipper 1950X (so amd64) port builds did not give >>>>> evidence of such problems based on the updated system. (Also >>>>> Optane media in a PCIe slot, also root on ZFS.) But the >>>>> errors seem rare enough to not be able to conclude much. >>>>=20 >>>> For aarch64 targeting aarch64 there was also this >>>> explicit corruption notice during the poudriere(-devel) >>>> bulk build: >>>>=20 >>>> . . . >>>> [CA72_ZFS] Extracting arm-none-eabi-gcc-8.4.0_3: ......... >>>> pkg-static: Fail to extract = /usr/local/libexec/gcc/arm-none-eabi/8.4.0/lto1 from package: Lzma = library error: Corrupted input data >>>> [CA72_ZFS] Extracting arm-none-eabi-gcc-8.4.0_3... done >>>>=20 >>>> Failed to install the following 1 package(s): = /packages/All/arm-none-eabi-gcc-8.4.0_3.pkg >>>> *** Error code 1 >>>> Stop. >>>> make: stopped in /usr/ports/sysutils/u-boot-orangepi-plus-2e >>>>=20 >>>> I'm not yet to the point of retrying after removing >>>> arm-none-eabi-gcc-8.4.0_3 : other things are being built. >>>=20 >>>=20 >>> Another context with my prior general update of /usr/ports/ >>> and the matching port builds: Back then I used USE_TMPFS=3Dall >>> but the failure is based on USE_TMPFS-"data" instead. So: >>> lots more I/O. >>>=20 >>=20 >> None of the 3 corruptions repeated during bulk builds that >> retried the builds that generated the files. All of the >> ports that failed by hitting the corruptions in what they >> depended on, built fine in teh retries. >>=20 >> For reference: >>=20 >> I'll note that, back when I was using USE_TMPFS=3Dall , I also >> did some separate bulk -a test runs, both aarch64 (Cortex-A72) >> native and Cortext-A72 targeting Cortex-A7 (armv7). None of >> those showed evidence of file corruptions. In general I've >> not had previous file corruptions with this system. (There >> was a little more than 245 GiBytes swap, which covered the >> tmpfs needs when they were large.) >=20 >=20 > I set up a contrasting test context and got no evidence of > corruptions in that context. (Note: the 3 bulk builds > total to around 24 hrs of activity for the 3 examples > of 460+ ports building.) So, for the Cortex-A72 system, I set up a UFS on Optane (U.2 via M.2 adapter) context and also got no evidence of corruptions in that context (same hardware and a copy of the USB3 SSD based system). The sequence of 3 bulks took somewhat over 18 hrs using the Optane. > root on UFS on portable USB3 SSD: no evidence of corruptions Also: root on UFS on Optane U.2 via M.2: no evidence of corruptions > vs.: > root on ZFS on optane in PCIe slot: solid evidence of 3 known = corruptions >=20 > Both had USE_TMPFS=3D"data" in use. The same system build > had been installed and booted for both tests. >=20 > The evidence of corruptions is rare enough for this not to > be determinative, but it is suggestive. >=20 > Unfortunately, ZFS vs. UFS and Optane-in-PCIe vs. USB3 are > not differentiated by this test result. >=20 > There is also the result that I've not seen evidence of > corruptions on the ThreadRipper 1950 X (amd64) system. > Again, not determinative, but suggestive, given how rare > the corruptions seem to be. So far the only things unique to the observed corruptions are: root on ZFS context (vs. root on UFS) and: Optane in a PCIe slot (but no contrasting ZFS case tested) The PCIe slot does not seem to me to be likely to be contributing. So this seem to be suggestive of a ZFS problem. A contributing point might be that the main [so: 14] system was built via -mcpu=3Dcortex-a72 for execution on a Cortext-A72 system. [I previously ran into a USB subsystem mishandling of keeping things coherent for the week memory ordering in this sort of context. That issue was fixed. But back then I was lucky enough to be able to demonstrate fails vs. works by adding an appropriate instruction to FreeBSD in a few specific places (more than necessary as it turned out). Someone else determined where the actual mishandling was that covered all required places. My generating that much information in this context seems unlikely.] =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)