From nobody Mon Aug 05 07:15:40 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wcnlv1DdFz5SjdJ for ; Mon, 05 Aug 2024 07:15:55 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-22.consmr.mail.gq1.yahoo.com (sonic317-22.consmr.mail.gq1.yahoo.com [98.137.66.148]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wcnlt5rKrz4gQ9 for ; Mon, 5 Aug 2024 07:15:54 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1722842153; bh=swk5YYSiygGmRUj0rCFqK2fzAQKERnT7C4F2RKWWfcA=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=Y54DBkF03sWxC4A+t5ldA6N7GImDsWepkqRZtIwCtA3opZ5Ca1hiBOoRbBI2GZQZFDEIUhr/O92f1W4TPIoh46CY/iXVAQDO38QrzE+5UZrXeOLSD/JITclBLseclcFlOMvYCzGjeC7w/VmcJdywxnDlSO5qChsaOyBQb0Zg28s8+vx8yzrO3OZ8+EhIRPSF17lIbCLN7NMtLzotwplq8o86zju8OyWI70RSxsQlgXZDimqcpsQFXi+E3kOnWxjTsd6GdaWRjmN0dtiKKewOXfJmc+pvZ59jrEHCoAZeECMc4OLvFA8yDt/QgrhAGMw+fgRxQm4uaW1vawSQMdMTGg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1722842153; bh=PSOE2s9DRg5Kqjs7WN3A2Jchlu+XWLtG2SXkWly/hPU=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=TeLo7frZf2xDo5MQN8l2QXwAB4XRXCI30sQ9C2JqraeutLxzQ3HFT1ZH3Qf9jGu1ich2o/skYTN3VEH18atXOgh18iji9eIAydmmF9uvckS3NzW2ZL21Pc4b5/VVQrRd5wwQjmUUtcwVkHsyrpBsxQSdUPiWeACdHvjQrmXWTLFBvQ0ovaobPS5zSN9ha7woB9eSAdKNLGw0B4h5X6wKm0pPhrFKyQTioIiQlzsDkwPpxQm68ze/hTl04hK0GExk/79xCVTUXb4QF2E87eUu9DrmOyX7jtm1Ouu96TAoc0qHUrKayDDnj6D//VKG95SkXlveq+DwJNjDxKylMI4Zeg== X-YMail-OSG: 8tNrN3gVM1mirgTkZxL6Ft9cpBcVEUFyNt5IIXn5GVi5sDdzYSqVD8XaujKzNjf qUK6eaqjHTUv..Tpyw_6qJqzsJuzhN8ySoT6nSVi_OzMPnxEQFckfn1JjRFmv_AI2wWLcIZQYp9l 5ukfbGUS6DRpDZ1FcBKct5fZXW0y8QcIFPPVOwqzZIYPZCzfStqXgB_tSiZUOld5ZQAqrRNOWf_K CAgbw8uTBeeKiXY6wwU26SQ0KG8ZbJhk..9WHcIWARTtEYtYE05EVRjZMeYJk0nmfi_HRtmRfNcK M1o6aArmvgBG5pt9gg2JUfjk3RKaGuBFI03pseuilR_BOp3dI8b_BE4TjsqI72gIpL2w97KRQOUz UGsghHtNANc40GPglWeiJCGiGnkKpe9aXsgVZSSQHFnWxZoGAsWzkKAV.GgHtVq9XLlxVnIuWmk0 MlZpAUjpTNKMYOAIWkx7tsrvPEmNAPjNydaZWadnY8aNbKtLn.l1M7ZE.j5FLy0Y..8RXtnWVt57 7IFl9ML0MPix4Y9JnRIOf.xSJyVUmiz1GJAGVxihJvg8iYnLHAoApItXi1ozahLgNgGKp4.IYTVZ z25Fk5fkIcB0lO8LbnVtfT5CM7xbOG7gsxP8r3a3F76HmIWb1KU7bteDBDQiWxX5u6xOxRO8Cgxz h.KV6IzjCwydBd3FIpIe4XD_LZiy52MIXeNJfQj34W18EuIyjoDniy9ah33CVT5ZONpJ8ikvSC6S YTBZPbh_WFS8pCd7KXCam.GCxVldQbj8WaZEwJmSqa0XfAxo9bYE94OfH3qkdcFN4ycPF1gmgfCW PrIEuPVu2d.rhPXZVIprVRFZ7Rd7Av1K0FBe7xS.VowjdAqbmfGObn_PkDtAJvf_IyMhsg0VczMv Zu7vPGyp.3uDz1jRuvhX9HQ6ZMwGndNxgGxTEupWCJaw52QrWpUmgVqycLo73Fu7UaX_2SVJ6H5F AEwx98YF927hl0yk5EXhCz12TCY3fOq3ykbtJqrhl5OXeGw3I9sEmD5Bi1zR8.wMs_pYm.Y.baRH OQblO.j15fAaKHyK0FsjEKExWtZaIpCQvnPpAXEf2G7z5XJ5b8Ygftuy1S_FtOC2VhyFCM2_TIBK YBFpfa9bxv5WCCI1y7aLVMQnelti3qcxHlVegtbv43T09WOHp6BvDgFOF4mbyVu9mXrw5DzVT4tO 87SQc5Ydsgj4My0landScxgY.BDaIH0uvxYZDhOPB21gilSGRc0BiC5jiQOxTX.ROG9x5nX.Hcvm 9XHlqR.ITMtmCkXgJgo04biyOGYmI95u3tMD8PLccT9.qLGkOTi809xRDgu8zMRUNc1HGU44jGpv TrqFb21rVYHeYyw7719Um4kapFcDUW5RfW5d5jgU5_8N3WVqQCzpoo1i9TG.o7JXrM8GK.kQ0aP8 ufBmpK8AjTZM4l7Llsvy9Uo7GW83H1obL2.IZn3D4u4YcToxpHSeaAmKc7zwpkb9KhiQSNWS0X3E dCLnVY8WsHKMf8gvwZrQYgwz3h4aIz_O7h08lasgebDHMFM3FBR3A11EMdceiAELx4np4Ezh..4. .0wjX00UhRNOVvxQEvNUOHexgl6kAhQuBTpBtxNiLcWGIPckyfsE188nnHNY6i1Q8BuY.hAmydUJ Vw1d4Hqwan28YlkSZahTNz1A6klWR7irXOekdtbenxdfHhBvJvlJ6sEwQgR6HniVrgLhDqZId3v0 gch5TpTTKhXZ8TEN.0K5Ul.F4QzBVhXLN2LAMAwg2eqkNtyO4FhOK4t5ANsYEk5GxYUmCEK.uR3I bttAnp6D1iDltPIU45jCrYLcCIUDpFVZhOsRsnCYV8lEB0cSW.RjmFlnD5SSkR8HTVIRJys5lkNL HOJcQnvhv_E330Act3w1nxZatoT3gHhxySkFUlyHzUDLPqsydrja1rL_abVaVS5zEjtQiXuFJ88m WgE.E5Wqvu35lXyKS70OZyI.vsS5eDUsggZj4XG4a_tuRkIzkqRk2cYiOe5DRLDYmRu52RPaxvs_ IF.nSVk84oJqJLbhsyGHyv6q9W5h3bFBI9gefKpFpOX4UcCm41TXmJSmrtqcuqjNVvHxNf_aBwMM 09EyCRgpewNR6VLmQ_pROjBDB_UT3mwYozaWdnV5tjFZCt.YnyyXCttasHUePcMXVstniSn8ije2 J2XCf5NMkjtIzeEZc4PSMW9KM38C4bWbbSSFcXvQoXindSyFUMTE6KIq_RRFFbYeXciSaJViu6oe SHy.4f8yV42A08Y6og0wRK6zhN0v49NqtJQekRMeiYB0hoITD4zC.ZBnWfIWQ7wuJFwNCO8i3vdc - X-Sonic-MF: X-Sonic-ID: e7391727-6347-4449-9d07-fa233211e59c Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Mon, 5 Aug 2024 07:15:53 +0000 Received: by hermes--production-gq1-5d95dc458-24x88 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 2ca5aa8b132b47836145f7a507b16d94; Mon, 05 Aug 2024 07:15:50 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: Any known way to build devel/llvm* ( such as devel/llvm19 ) with --threads=1 for its linker activity during the build? From: Mark Millard In-Reply-To: Date: Mon, 5 Aug 2024 00:15:40 -0700 Cc: FreeBSD Toolchain , FreeBSD ARM List Content-Transfer-Encoding: quoted-printable Message-Id: References: <4FFD603F-E67C-4B62-B91B-8BE365EAA050@yahoo.com> <82E78798-C376-45C4-80FE-96AD14229419@yahoo.com> To: mmel@freebsd.org X-Mailer: Apple Mail (2.3774.600.62) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4Wcnlt5rKrz4gQ9 On Aug 4, 2024, at 22:53, Michal Meloun wrote: > On 04.08.2024 23:31, Mark Millard wrote: >> On Aug 3, 2024, at 23:07, Mark Millard wrote: >>> My recent attempts to build devel/llvm18 and devel/llvm19 in an = armv7 context (native or aarch64-as-armv7) have had /usr/bin/ld failures = that stop the build and report as: >>>=20 >>> LLVM ERROR: out of memory >>> Allocation failed >>>=20 >>> (no system OOM activity or notices, so just a process = size/fragmentation issue, or so I would expect). >>>=20 >>> On native armv7 I also had rust 1.79.0 fail that way so --but = aarch64-as-armv7 built it okay. >>>=20 >>> I'm curious if --threads=3D1 use for the linker might allow the = devel/llvm* builds to complete at this point. Similarly for rust. (top = showed that the ld activity was multi-threaded.) >>>=20 >>> Note: The structure of the poudriere-devel based native build = attempts is historical and it used to work. Similarly for the = aarch64-as-armv7 based build attempts. For now I'd just be exploring = changes that might allow much of my historical overall structure to = still work. But I expect that things are just growing to the point = building is starting to be problematical with process address spaces = that are bounded by a limit somewhat under 4 GiBytes. >>>=20 >>>=20 >>> Native armv7 was a 2 GiByte OrangePi+ 2ed (4 cores) that had >>> at boot time: >>>=20 >>> AVAIL_RAM+SWAP =3D=3D 1958Mi+3685Mi =3D=3D 5643Mi >>>=20 >>> and later had "Max(imum)Obs(erved)" figures: >>>=20 >>> Mem: . . ., >>> 1728Mi MaxObsActive, 275192Ki MaxObsWired, 1952Mi = MaxObs(Act+Wir+Lndry) >>>=20 >>> Swap: 3685Mi Total, . . ., >>> 1535Mi MaxObsUsed, 3177Mi MaxObs(Act+Lndry+SwapUsed), >>> 3398Mi MaxObs(A+Wir+L+SU), 3449Mi (A+W+L+SU+InAct) >>>=20 >>>=20 >>> The aarch64-as-armv7 was a Win DevKit 2023 that has 8 cores and: >>>=20 >>> AVAIL_RAM+SWAP =3D=3D 31311Mi+120831Mi =3D=3D 152142Mi >>>=20 >>> So lots of 4 GiByte or smaller processes would fit. >>>=20 >> Absent finding a way to get --threads=3D1 to be what is used, I >> made the following crude way to test, built it, installed it >> in the armv7 directory tree used for aarch64-as-armv7, and >> then started an aarch64-as-armv7 test of building devel/llvm19 >> to see what the consequences are (leading whitespace details >> might not be preserved): >> # git -C /usr/main-src/ diff contrib/llvm-project/ >> diff --git a/contrib/llvm-project/lld/ELF/Driver.cpp = b/contrib/llvm-project/lld/ELF/Driver.cpp >> index 8b2c32b15348..299daf7dd6fa 100644 >> --- a/contrib/llvm-project/lld/ELF/Driver.cpp >> +++ b/contrib/llvm-project/lld/ELF/Driver.cpp >> @@ -1587,6 +1587,9 @@ static void readConfigs(opt::InputArgList = &args) { >> arg->getValue() + "'"); >> parallel::strategy =3D hardware_concurrency(threads); >> config->thinLTOJobs =3D v; >> + } else if (sizeof(void*) <=3D 4) { >> + log("set maximum concurrency to 1, specify --threads=3D to = change"); >> + parallel::strategy =3D hardware_concurrency(1); >> } else if (parallel::strategy.compute_thread_count() > 16) { >> log("set maximum concurrency to 16, specify --threads=3D to = change"); >> parallel::strategy =3D hardware_concurrency(16); >> Basically, if the process address space has to be "small", avoid >> any default memory use tradeoffs that multi-threading the linker >> might involve --even if that means taking more time. >> We will see if: >> [00:00:33] [07] [00:00:00] Building devel/llvm19@default | = llvm19-19.1.0.r1 >> still fails to build as armv7 vs. if the change leads it to >> manage to build as armv7. >> =3D=3D=3D >> Mark Millard >> marklmi at yahoo.com >=20 > I can build llvm18 and rust 1.79 on native armv7 without problems - = on Tegra TK1, without poudriere and on the ufs filesystem. IMHO = poudriere is unusable on 32bit systems. On Windows DevKit 2023 in a armv7 chroot I can build rust 1.79.0 as well. I've not tried a recent devel/llvm18 in that context, just devel/llvm19 . An armv7 process in this context can use about 1 GiByte more memory space than on the OrangePi+ 2ed. (See later program example outputs.) Previously, devel/llvm18-18.1.7 had built fine some time back. So I'm trying the modern 18.1.8_1 now on the Windows DevKit 2023. But this is with forcing of --threads=3D1 for lld: same context as the recent devel/llvm19 exploration. Note: UFS context, not ZFS. How does the Tegra TK1 context compare for the following program and the example command? OrangePi+ 2ed (so: armv7 native with 2 GiBytes of RAM): # more process_size.c // cc -std=3Dc11 process_size.c // ./a.out 268435456 268435456 268435456 268435456 268435456 268435456 = 268435456 268435456 268435456 268435456 268435456 268435456 268435456 = 134217728 67108864 33554432 16777216 8388608 4194304 2097152 1048576 #include #include #include #include #include int main(int argc, char *argv[]) { size_t totalsize=3D 0u; for (int i =3D 1; i < argc; ++i) { errno =3D 0; size_t size =3D strtoul(argv[i],NULL,0); void *p =3D malloc(size); if (p) totalsize +=3D size; printf("malloc(%zu) =3D %p [errno =3D %d]\n", size, p, errno); } printf("approx. total, a lower bound: %zu MiBytes\n", = totalsize/1024u/1024u); return 0; } # cc -std=3Dc11 process_size.c # ./a.out 268435456 268435456 268435456 268435456 268435456 268435456 = 268435456 268435456 268435456 268435456 268435456 268435456 268435456 = 134217728 67108864 33554432 16777216 8388608 4194304 2097152 1048576 malloc(268435456) =3D 0x20800180 [errno =3D 0] malloc(268435456) =3D 0x30801980 [errno =3D 0] malloc(268435456) =3D 0x40802640 [errno =3D 0] malloc(268435456) =3D 0x50803600 [errno =3D 0] malloc(268435456) =3D 0x608048c0 [errno =3D 0] malloc(268435456) =3D 0x70805140 [errno =3D 0] malloc(268435456) =3D 0x80806580 [errno =3D 0] malloc(268435456) =3D 0x90807780 [errno =3D 0] malloc(268435456) =3D 0xa0808700 [errno =3D 0] malloc(268435456) =3D 0x0 [errno =3D 12] malloc(268435456) =3D 0x0 [errno =3D 12] malloc(268435456) =3D 0x0 [errno =3D 12] malloc(268435456) =3D 0x0 [errno =3D 12] malloc(134217728) =3D 0xb0809a00 [errno =3D 0] malloc(67108864) =3D 0x0 [errno =3D 12] malloc(33554432) =3D 0xb880a5c0 [errno =3D 0] malloc(16777216) =3D 0xba80b0c0 [errno =3D 0] malloc(8388608) =3D 0x0 [errno =3D 12] malloc(4194304) =3D 0x0 [errno =3D 12] malloc(2097152) =3D 0xbb80c180 [errno =3D 0] malloc(1048576) =3D 0xbba0de80 [errno =3D 0] approx. total, a lower bound: 2483 MiBytes Same program with same command on Windows DevKit 2023 in armv7 chroot (aarch64-as-armv7 with 32 GiBytes of RAM): # ./a.out 268435456 268435456 268435456 268435456 268435456 268435456 = 268435456 268435456 268435456 268435456 268435456 268435456 268435456 = 134217728 67108864 33554432 16777216 8388608 4194304 2097152 1048576 malloc(268435456) =3D 0x20800b00 [errno =3D 0] malloc(268435456) =3D 0x30801600 [errno =3D 0] malloc(268435456) =3D 0x40802cc0 [errno =3D 0] malloc(268435456) =3D 0x50803c80 [errno =3D 0] malloc(268435456) =3D 0x608042c0 [errno =3D 0] malloc(268435456) =3D 0x70805b00 [errno =3D 0] malloc(268435456) =3D 0x808063c0 [errno =3D 0] malloc(268435456) =3D 0x90807580 [errno =3D 0] malloc(268435456) =3D 0xa0808b40 [errno =3D 0] malloc(268435456) =3D 0xb0809980 [errno =3D 0] malloc(268435456) =3D 0xc080abc0 [errno =3D 0] malloc(268435456) =3D 0xd080ba00 [errno =3D 0] malloc(268435456) =3D 0xe080cc80 [errno =3D 0] malloc(134217728) =3D 0xf080d700 [errno =3D 0] malloc(67108864) =3D 0x0 [errno =3D 12] malloc(33554432) =3D 0xf880eb40 [errno =3D 0] malloc(16777216) =3D 0xfa80fc00 [errno =3D 0] malloc(8388608) =3D 0x0 [errno =3D 12] malloc(4194304) =3D 0xfb810840 [errno =3D 0] malloc(2097152) =3D 0xfbc117c0 [errno =3D 0] malloc(1048576) =3D 0xfbe12940 [errno =3D 0] approx. total, a lower bound: 3511 MiBytes Note: If the Tegra TK1 in question has more than 4 GiBytes of RAM, the command line should explore more than the example that I used. Note: I've used the program for other patterns of allocations. That is why it is not just a fixed exploration algorithm. As for poudriere-devel, I find it useful, even on the OrangePi+ 2ed. But mostly that is a rare run that is checking on how well the handling goes for the 2 GiByte of RAM context (with notable SWAP for the size of RAM). In other words, monitoring the growth in a context that will break sooner than my other contexts generally would. The tests take days overall, most of the time being for rust and a llvm* . Historically I've been able to have 2 builders, each with MAKE_JOBS_NUMBER_LIMIT=3D2 , so all 4 cores in use building lang/rust and a devel/llvm* at the same time successfully in poudriere-devel on the 2 GiByte OrangePi+ 2ed. (This was before recently imposing --threads=3D1 experiments, given the recent build failures.) =3D=3D=3D Mark Millard marklmi at yahoo.com