From nobody Mon Aug 05 07:42:01 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WcpLM2w6vz5Sm1L for ; Mon, 05 Aug 2024 07:42:19 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-19.consmr.mail.gq1.yahoo.com (sonic305-19.consmr.mail.gq1.yahoo.com [98.137.64.82]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WcpLL24W9z4kbP for ; Mon, 5 Aug 2024 07:42:18 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=e8g5gOaj; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.82 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1722843736; bh=NYwAxZmSzL98MXgzGLJsVvOTHvAlhyA5vkl3OY+v7ug=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=e8g5gOajnCFgnzA9OHS6mv5c0HzWK6iVHlt7fQvUTva/RVrSiLfzixbTq+hjV5CvoYdGlLLxFbmOFiQrT9hlPpqcyCvWuY0ZevSnRHsAaz4buRwuQFPKEH+/fgqur7FQ2JMBCDWRJCXJiH85LlETxPi9Czk7o1/YsyjiGglfa7FBwrn+xOqTNYVyU/edhoePw+/x7G6BhYF4tE+RVDSTWmueI7hbGfgIZ1XnUwydjhPc9xQAfI2dhK3kO4bdhnJ9C4oD01Gw52/LOHiYobmti6AwEsNTxjLc1FTdHnzXD2xi34IJE6GjHNiqwh7I0QjrHOn9QNq0DO9VVEN0t0widA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1722843736; bh=fayJwHjHP0ddT9Eg8/ABmG3rw97T4iZfx+Qijk+gtbp=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=XssWPI5J2UTPO13MNxdPXobPScpeQFXiSq1fY5dyin1gMeAA8DlCtkkPkjFSC37WFQvvCcBTmtDYr4E7lsl9rXKqViYyQ3vDFJAIS8haoJ9czZmcMbepMloU52XCOgrKR3ZYIMdzmqfPq+cbQWCJ9WnvJd6fPzSwNY/uBYWCNCxMqN9g2Zd85KOQCVcKQgZST0pXpDlUZe+wm8luDd46dvMWAmOqlsfCm/EeVMzPJYjyupBhuDDpQDDZxZUw6k9vmeUWBhyRcnUww9Fy1d+zpO6y1g1TVfQAmtPEFWnkRrQngiWdujqmKK8wcQ8nVsCpqFQ++Q8hWhs17x7p914JSw== X-YMail-OSG: ieAL2c0VM1l_SLMsl1n._FqnsSlJFBBYi.F5wN_gzzBOjThDYCGe6xJKithRBCh 0U9ba6HTOPYo8pFaz7fK9brXKowBhq.OwYY2BsmP3UENGJn6h90hs8qhZh.Yebtd789YgDZMVlsd RzAZRzLnNNsF2rLQYXa0XIJlxhNldKhQZr.SUdZae.iDU.bYhrkiP6honDnVkqxkV07JuCIAB.b4 OfvV.7_o0.psUByTlkKmfl2G.p_.G5FwETIYtd.xz2j60mREF6UnEVhyjpjECKc4GcFokZ3YS_nd N8s8DitXBi13g4g8Uz4Ijn3aftnOCMh3KFc8_Uqm2Sho59RWipb8zBASdSmti_ejeJkFoNfJOF6p z0iFj8PoaT5wVT3MPtJYGEN.NJxoMxgwN2O3y9frGYCDJGsiPR.d6zwgO4XHrQIJjOxDpee2ZeHg NsaiXJyZ9a2xiXVDO9N0rx5rVzslzazsIwjHwLIsMlSFtXwG8ugl0cjlVY7ALiHi_BiVXFAc.HSu Wf0TWG8nTPXl03gG_I69TRI6L7LN6F8ibGH73kqnh7xIVimhoAXH6bjtP4uYxxcq.EXCijaL5wVH gyXSeIYjwMDqKj6sl0.NJlaE66yOr.6nONjzZKc3ZYF7GyH_wWiFK3CeNybbb1KE5F7AdjfifuBC pnZvSNgc5pU93GO8xpBCNpEInJN73poTJ7r7WjuklkV6b36o7WqHBuanbzsQKSVtI3CjmbyQ6xdS xBTXQxE0N38At5SxyXnl2qbH6R1oHn0HtnCPK3kYjx61z2Qw38EOosxHFQMY5mScrnU0B0cHH.qM t3XJPh4Z33qQFUN8BAmVc.wKwOvm2YUXF5ZBVYkYGxVy28mjvsTbVceH0XetxiRIf3T5u7QyetOy ienz7taRp0b.l0Y1MT7HonrQF.4ImPngWzapfce3EazMaDC6ioCYlNjDZYQy5F1IN.c8Kzylxnw1 .WZ0bPWj.0b3xe1ZQySGAz5Xoh9RvZCBOktqjLx5jNBG_Qm.08f8gbbqZGZiUOhnDwGghX_T7FYg sPmKSSqWU9PgOyS21eXO8LF.x_WiuzIqusXQJZhB3DsvgYo7kHrAJ31PgagqujbkaNjTD14VsYAP 9irLy56hix_Och.irFO25nzvpiO601jOc7PFxIBEoHiXswGLjqPgZxgZEwEuBUYINitfBUcDg8HV B.qNCQnIvudu8Sal8wp4zzh5rfCHEKlxjIzsolw03DNCF6KUcrreVY.DKl2FZO1Y_ctcZUJiUZrv e5CveF4VB3hSaJGpr1_5y5E_DpS8G79xhpTCIPzTvePGucq8AMukeFgKkIoCWXhu3u6WZ4vVTaGs WRuUmpCtDdRLmB4dbuYotE0mfanNEeAThl_J0Dr7tiOah9K5wx_OP07rcTGcLoc.OrIVGchLFuc6 x07TAkfiaKacB3XX3vXv6Ng2ASquxs7PUWrmmQLESf1.VuZUARePPO0.4J9_CdjwiUjF_Dg0jnNm UYyYrTiW9aQlxfSqJ9ikLnvLK5dd2luLOheNDh4JHWeVQADzBwbiSaYzsUCOG0vgUtjL3x74mzHZ lR02SRiN0KtZI.6Nzgiis.EeHwkc4MGEwze0PNtf4irT5sipBqsV4p8BQ8v0NsMwdvqJlpzUEPvv kZp3Rqddwo0BTfnRhaZmJ5yNsaNgibygNgDrI9_qYe9hUcegPZgvX4PyYP1qVP7IASRIO7BEatKa 7tY5whd110LRIfpCHR2SbXcRtSSaQ4XevwYpJbLP6coh1HubHkJhqf0bViTOn3.AFLcY12C.gZ_4 Di7h6uCoBRLlp3Yk4otHrd4dxl0G6qWHMy4mQjgJxvYpM.Hs2o04ogHUD36mB9tqhCorlK12z7po dkN2TmDKxczHRO66ecwk3pCYwgyrcnxJdDs9aFo9.u.dLZZW2l5NSjL1GT4e.Dq4eXLaVS9SaIwr Y8ISyXRiBkA6F_B8FP1.5bAqXXXUV38FYaQmjlTQLt1toCfQOk9bKJM7DdgpZQBXQabvo5ieAD.7 Ozke4GiQAED6zR8oFlXkFyUyWGscQsHi8VteC44zDJfYQbNz3MIfYU4XKAS7bxZvITaYg.MBw3wM dwiTjwezvVX0vlUOcexjyRgFG6_WOzR67WnTcqW8b29.tUp_3_Es3hWYq_nMVlSvk4Fq5YADgGHP zO5j.d0eKY8LjWCJsvgCxpJALjefxwCWZA.KK1JzMSuWjVFW4vIZeTR_82fwIwrcZOUKsW9l92Ai esla44Xgp9JbVyXQCr7JPUpdLXGKDL4YsfbykQvsaWb68mZWYla4qbWUW9s8_k7D6VyVVa08svyE - X-Sonic-MF: X-Sonic-ID: c89f68f0-970a-4bde-8634-849768c4d7f9 Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Mon, 5 Aug 2024 07:42:16 +0000 Received: by hermes--production-gq1-5d95dc458-kk28l (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 3baa033702cd98f2c46b23c0c62fac3d; Mon, 05 Aug 2024 07:42:11 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: Any known way to build devel/llvm* ( such as devel/llvm19 ) with --threads=1 for its linker activity during the build? From: Mark Millard In-Reply-To: Date: Mon, 5 Aug 2024 00:42:01 -0700 Cc: FreeBSD Toolchain , FreeBSD ARM List Content-Transfer-Encoding: quoted-printable Message-Id: References: <4FFD603F-E67C-4B62-B91B-8BE365EAA050@yahoo.com> <82E78798-C376-45C4-80FE-96AD14229419@yahoo.com> To: mmel@freebsd.org X-Mailer: Apple Mail (2.3774.600.62) X-Spamd-Bar: -- X-Spamd-Result: default: False [-3.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.998]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; RCPT_COUNT_THREE(0.00)[3]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.82:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.82:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] X-Rspamd-Queue-Id: 4WcpLL24W9z4kbP On Aug 5, 2024, at 00:27, Mark Millard wrote: > On Aug 5, 2024, at 00:15, Mark Millard wrote: >=20 >> On Aug 4, 2024, at 22:53, Michal Meloun = wrote: >>=20 >>> On 04.08.2024 23:31, Mark Millard wrote: >>>> On Aug 3, 2024, at 23:07, Mark Millard wrote: >>>>> My recent attempts to build devel/llvm18 and devel/llvm19 in an = armv7 context (native or aarch64-as-armv7) have had /usr/bin/ld failures = that stop the build and report as: >>>>>=20 >>>>> LLVM ERROR: out of memory >>>>> Allocation failed >>>>>=20 >>>>> (no system OOM activity or notices, so just a process = size/fragmentation issue, or so I would expect). >>>>>=20 >>>>> On native armv7 I also had rust 1.79.0 fail that way so --but = aarch64-as-armv7 built it okay. >>>>>=20 >>>>> I'm curious if --threads=3D1 use for the linker might allow the = devel/llvm* builds to complete at this point. Similarly for rust. (top = showed that the ld activity was multi-threaded.) >>>>>=20 >>>>> Note: The structure of the poudriere-devel based native build = attempts is historical and it used to work. Similarly for the = aarch64-as-armv7 based build attempts. For now I'd just be exploring = changes that might allow much of my historical overall structure to = still work. But I expect that things are just growing to the point = building is starting to be problematical with process address spaces = that are bounded by a limit somewhat under 4 GiBytes. >>>>>=20 >>>>>=20 >>>>> Native armv7 was a 2 GiByte OrangePi+ 2ed (4 cores) that had >>>>> at boot time: >>>>>=20 >>>>> AVAIL_RAM+SWAP =3D=3D 1958Mi+3685Mi =3D=3D 5643Mi >>>>>=20 >>>>> and later had "Max(imum)Obs(erved)" figures: >>>>>=20 >>>>> Mem: . . ., >>>>> 1728Mi MaxObsActive, 275192Ki MaxObsWired, 1952Mi = MaxObs(Act+Wir+Lndry) >>>>>=20 >>>>> Swap: 3685Mi Total, . . ., >>>>> 1535Mi MaxObsUsed, 3177Mi MaxObs(Act+Lndry+SwapUsed), >>>>> 3398Mi MaxObs(A+Wir+L+SU), 3449Mi (A+W+L+SU+InAct) >>>>>=20 >>>>>=20 >>>>> The aarch64-as-armv7 was a Win DevKit 2023 that has 8 cores and: >>>>>=20 >>>>> AVAIL_RAM+SWAP =3D=3D 31311Mi+120831Mi =3D=3D 152142Mi >>>>>=20 >>>>> So lots of 4 GiByte or smaller processes would fit. >>>>>=20 >>>> Absent finding a way to get --threads=3D1 to be what is used, I >>>> made the following crude way to test, built it, installed it >>>> in the armv7 directory tree used for aarch64-as-armv7, and >>>> then started an aarch64-as-armv7 test of building devel/llvm19 >>>> to see what the consequences are (leading whitespace details >>>> might not be preserved): >>>> # git -C /usr/main-src/ diff contrib/llvm-project/ >>>> diff --git a/contrib/llvm-project/lld/ELF/Driver.cpp = b/contrib/llvm-project/lld/ELF/Driver.cpp >>>> index 8b2c32b15348..299daf7dd6fa 100644 >>>> --- a/contrib/llvm-project/lld/ELF/Driver.cpp >>>> +++ b/contrib/llvm-project/lld/ELF/Driver.cpp >>>> @@ -1587,6 +1587,9 @@ static void readConfigs(opt::InputArgList = &args) { >>>> arg->getValue() + "'"); >>>> parallel::strategy =3D hardware_concurrency(threads); >>>> config->thinLTOJobs =3D v; >>>> + } else if (sizeof(void*) <=3D 4) { >>>> + log("set maximum concurrency to 1, specify --threads=3D to = change"); >>>> + parallel::strategy =3D hardware_concurrency(1); >>>> } else if (parallel::strategy.compute_thread_count() > 16) { >>>> log("set maximum concurrency to 16, specify --threads=3D to = change"); >>>> parallel::strategy =3D hardware_concurrency(16); >>>> Basically, if the process address space has to be "small", avoid >>>> any default memory use tradeoffs that multi-threading the linker >>>> might involve --even if that means taking more time. >>>> We will see if: >>>> [00:00:33] [07] [00:00:00] Building devel/llvm19@default | = llvm19-19.1.0.r1 >>>> still fails to build as armv7 vs. if the change leads it to >>>> manage to build as armv7. >>>> =3D=3D=3D >>>> Mark Millard >>>> marklmi at yahoo.com >>>=20 >>> I can build llvm18 and rust 1.79 on native armv7 without problems - = on Tegra TK1, without poudriere and on the ufs filesystem. IMHO = poudriere is unusable on 32bit systems. >>=20 >> On Windows DevKit 2023 in a armv7 chroot I can build rust 1.79.0 >> as well. I've not tried a recent devel/llvm18 in that context, >> just devel/llvm19 . An armv7 process in this context can use >> about 1 GiByte more memory space than on the OrangePi+ 2ed. (See >> later program example outputs.) >>=20 >> Previously, devel/llvm18-18.1.7 had built fine some time back. >> So I'm trying the modern 18.1.8_1 now on the Windows DevKit 2023. >> But this is with forcing of --threads=3D1 for lld: same context as >> the recent devel/llvm19 exploration. >>=20 >> Note: UFS context, not ZFS. >>=20 >> How does the Tegra TK1 context compare for the following >> program and the example command? >>=20 >> OrangePi+ 2ed (so: armv7 native with 2 GiBytes of RAM): >>=20 >> # more process_size.c >> // cc -std=3Dc11 process_size.c >> // ./a.out 268435456 268435456 268435456 268435456 268435456 = 268435456 268435456 268435456 268435456 268435456 268435456 268435456 = 268435456 134217728 67108864 33554432 16777216 8388608 4194304 2097152 = 1048576 >>=20 >> #include >> #include >> #include >> #include >> #include >>=20 >> int main(int argc, char *argv[]) >> { >> size_t totalsize=3D 0u; >> for (int i =3D 1; i < argc; ++i) { >> errno =3D 0; >> size_t size =3D strtoul(argv[i],NULL,0); >> void *p =3D malloc(size); >> if (p) totalsize +=3D size; >> printf("malloc(%zu) =3D %p [errno =3D %d]\n", size, p, errno); >> } >> printf("approx. total, a lower bound: %zu MiBytes\n", = totalsize/1024u/1024u); >> return 0; >> } >> # cc -std=3Dc11 process_size.c >> # ./a.out 268435456 268435456 268435456 268435456 268435456 268435456 = 268435456 268435456 268435456 268435456 268435456 268435456 268435456 = 134217728 67108864 33554432 16777216 8388608 4194304 2097152 1048576 >> malloc(268435456) =3D 0x20800180 [errno =3D 0] >> malloc(268435456) =3D 0x30801980 [errno =3D 0] >> malloc(268435456) =3D 0x40802640 [errno =3D 0] >> malloc(268435456) =3D 0x50803600 [errno =3D 0] >> malloc(268435456) =3D 0x608048c0 [errno =3D 0] >> malloc(268435456) =3D 0x70805140 [errno =3D 0] >> malloc(268435456) =3D 0x80806580 [errno =3D 0] >> malloc(268435456) =3D 0x90807780 [errno =3D 0] >> malloc(268435456) =3D 0xa0808700 [errno =3D 0] >> malloc(268435456) =3D 0x0 [errno =3D 12] >> malloc(268435456) =3D 0x0 [errno =3D 12] >> malloc(268435456) =3D 0x0 [errno =3D 12] >> malloc(268435456) =3D 0x0 [errno =3D 12] >> malloc(134217728) =3D 0xb0809a00 [errno =3D 0] >> malloc(67108864) =3D 0x0 [errno =3D 12] >> malloc(33554432) =3D 0xb880a5c0 [errno =3D 0] >> malloc(16777216) =3D 0xba80b0c0 [errno =3D 0] >> malloc(8388608) =3D 0x0 [errno =3D 12] >> malloc(4194304) =3D 0x0 [errno =3D 12] >> malloc(2097152) =3D 0xbb80c180 [errno =3D 0] >> malloc(1048576) =3D 0xbba0de80 [errno =3D 0] >> approx. total, a lower bound: 2483 MiBytes >>=20 >>=20 >> Same program with same command on Windows DevKit 2023 in >> armv7 chroot (aarch64-as-armv7 with 32 GiBytes of RAM): >>=20 >> # ./a.out 268435456 268435456 268435456 268435456 268435456 268435456 = 268435456 268435456 268435456 268435456 268435456 268435456 268435456 = 134217728 67108864 33554432 16777216 8388608 4194304 2097152 1048576 >> malloc(268435456) =3D 0x20800b00 [errno =3D 0] >> malloc(268435456) =3D 0x30801600 [errno =3D 0] >> malloc(268435456) =3D 0x40802cc0 [errno =3D 0] >> malloc(268435456) =3D 0x50803c80 [errno =3D 0] >> malloc(268435456) =3D 0x608042c0 [errno =3D 0] >> malloc(268435456) =3D 0x70805b00 [errno =3D 0] >> malloc(268435456) =3D 0x808063c0 [errno =3D 0] >> malloc(268435456) =3D 0x90807580 [errno =3D 0] >> malloc(268435456) =3D 0xa0808b40 [errno =3D 0] >> malloc(268435456) =3D 0xb0809980 [errno =3D 0] >> malloc(268435456) =3D 0xc080abc0 [errno =3D 0] >> malloc(268435456) =3D 0xd080ba00 [errno =3D 0] >> malloc(268435456) =3D 0xe080cc80 [errno =3D 0] >> malloc(134217728) =3D 0xf080d700 [errno =3D 0] >> malloc(67108864) =3D 0x0 [errno =3D 12] >> malloc(33554432) =3D 0xf880eb40 [errno =3D 0] >> malloc(16777216) =3D 0xfa80fc00 [errno =3D 0] >> malloc(8388608) =3D 0x0 [errno =3D 12] >> malloc(4194304) =3D 0xfb810840 [errno =3D 0] >> malloc(2097152) =3D 0xfbc117c0 [errno =3D 0] >> malloc(1048576) =3D 0xfbe12940 [errno =3D 0] >> approx. total, a lower bound: 3511 MiBytes >>=20 >>=20 >> Note: If the Tegra TK1 in question has more than >> 4 GiBytes of RAM, the command line should explore >> more than the example that I used. >>=20 >>=20 >> Note: I've used the program for other patterns of >> allocations. That is why it is not just a fixed >> exploration algorithm. >>=20 >>=20 >> As for poudriere-devel, I find it useful, even on >> the OrangePi+ 2ed. But mostly that is a rare run >> that is checking on how well the handling goes for >> the 2 GiByte of RAM context (with notable SWAP for >> the size of RAM). In other words, monitoring the >> growth in a context that will break sooner than >> my other contexts generally would. The tests take >> days overall, most of the time being for rust and >> a llvm* . >>=20 >> Historically I've been able to have 2 builders, >> each with MAKE_JOBS_NUMBER_LIMIT=3D2 , so all 4 >> cores in use building lang/rust and a devel/llvm* >> at the same time successfully in poudriere-devel >> on the 2 GiByte OrangePi+ 2ed. (This was before >> recently imposing --threads=3D1 experiments, >> given the recent build failures.) >=20 > I should have noted that my normal devel/llvm* builds > on aarch64 and armv7 avoid building: BE_AMDGPU and > MLIR . They also target BE_NATIVE instead of > BE_STANDARD . (aarch64 BE_NATIVE includes armv7 as > well.) Looking around, I see that my Windows DevKit 2023 context still has /etc/sysctl.conf containing: # Help armv7 effectively have more address space: kern.maxssiz=3D67108864 kern.maxdsiz=3D536870912 That actually dates back to before some related commit(s) were done for the armv7 process size issue --and might not be useful any more. =3D=3D=3D Mark Millard marklmi at yahoo.com