From nobody Tue Aug 06 18:27:21 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WdhcS6Wrzz5SQbL for ; Tue, 06 Aug 2024 18:27:36 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic312-24.consmr.mail.gq1.yahoo.com (sonic312-24.consmr.mail.gq1.yahoo.com [98.137.69.205]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WdhcS3Jxbz4Dpd for ; Tue, 6 Aug 2024 18:27:36 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1722968854; bh=9Hu8Ar0omMFoTuxDvH8Fk2FI2RtioRzPbUxIfUnaKJ8=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=Ty0JWSKqNX7Dgdf8rlPcyEovMLXGBwzwjHbRR2G/vkYvAp/4wluiZ7ukLMPWp6uQuVbLqHILprKGqt8DSCYYbWy0lZAjlK9XlpbOCQg5nMvc3dtQs6FV0x+OZA0C7Z1Q3UO6WIgNPFORDITFxycJUonTk5f5yOBeek7xuqDClLCvpiVod60Ak3irCnYUh3AJ3amz1WOTNUgPgp5xiYV9Nx4lTSKweiAOxevTfZf/zatXJ+7naJ15dwItnBnBOXIoc44hf1QSjsB14U2IiF7S3Ji+qGdZmrPfx87MEJQjn4iMxvW/EZVSmty99HYKVcyiH+qEF1lUq7lTRf/qgNamDQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1722968854; bh=kaIb9WIBubi2S6vwH2BQhEWLzI3gawyZSmo5n2ySwMj=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=Jf4q3FhclBvlT/nqxukE3zNEHztvP29Q/6a0ANm7Tv8hP2GcddnADPlqYKMDbfhRwXr1E9lFdY6RSY6FxUXmM/OtTm8TFtFz52/4HAYOqtyFr/Q2YeH1BDZqDcM3MNmdvs2w2gHKU1HTOE7bF1uEoteZTE7CahYBv98uOrVcR14myJchkssG5RTOspnLZ6De7y/vKo409OHhPccmQPxjbUNjA5GSI849tWB3EcSEK1XksJlb9ZIQH8lzXYGNtLkQ/PmOdaLmfOB4WwUn2oh6BpvkgRsEqlYBBOiFQTYQma2CrooSh2kPwf4Towp5gVRQC0YT4OlzTF4nHxMDL61aRw== X-YMail-OSG: NC8kTDcVM1kTP42JePbSkFJ4nllcY6ZKCHyckCIVauBaGZvUeH0J_iS_aXNahga ataoXDE_u5OXsjUdpXJ4RN8_EvWLX5xq8JCl1CATP1dw_dZPlcDILI6TmKZ4R9JymtvXSt5Y6YQp lLT__6F318BHaKHSTk0GbX6RPy9qSogWigvb39H8dB33DkRf5u73._ooo5ThgTBcb4Ga.flP5x5s 5sxzhVHwC8RazKqUDyinKuhiYg8uysSkmHZ.w0UXHRvOvif2qwd6gVvg8G7q4YPX12AP2hGllpyd vhnj_PyUz52.7Z92c8764yjw_KauRJXPcFD0jS._nT1ToMWqMHioFcbUN.PAYCyOJcDwzehEoFOr MVW7ifnuT8OARsUfZM5kmV.eq.JPsjooTDf6JKriHKgA3mrwP3qWpz5_OoiQOAZfOFDignRQ_uES Bx2.9DeOZf_WueC68S.Kw5i6z5keg9SEansvoMZ3LVaJG9mHNc.ElDv.MRUd_8wGHReF_uBsvuU8 R6yWooQK4ODL_1VczTjJVBkjQ0tXwjrgE72wWeyWb9SwDNM9tEk.FhjKRfGksUNp5j1md66rX_8r j4AEhjXDojHxkN5ipePq.YOmm1ABz6Fg6Es2FukumH_EakgZPMjNUYXgH1vHSV3.kmZOC840daqI esQWMBIS2onKvMWk2UhmZgzAUhObBGOreOnTN6qj2lxoX6NhEspNXsD2FhDpBsZaAq__XnUkjeu0 eDAV1duIrVWfZd2h3d7nXcm709U.dizaKHbbEk5jPs1ZP6nLq8Bk8DKuh1uZsOA6E1Ra0Bv5i5dn yhpSww5MLi83z.NWMZ7NbB_Tmyq25Qr8bCTbuqERieVTRIhSWdjD44kxGOffHHEA7g8IrTl0sOXa aW7UzWIpWQaYstOyLdJTFVaMCD04m2ZSZeXqLjgoxFBLA1IquYpOjYJuYXpbsMx71ilhTuwOqKci Gj_WCI1vkup4etSNv8p35R.zF8uxDb0iZLb7e1nNxV9AYW.Q_s6NOl6qyEbZ_JAMbks7A0kipHZr Mh3ZfpNLFi28A1ARnXxlghPGSctbEhtFn7c7LybmNCcwyzbZOWSxCiHgrikO7hOJQG.31qpyhtAE A4aTXsScWHWqpKaybMmb.zOoHehc5SqAhb2HKcvqLyvM9w0elaU.iSJhP8V_BYi3Dz1X64aMQUYk pru.rYv5jjBocxQRv4x4vuR8P_ZuwTYVkABvsO0uHofPyd36TYVxf_K4k5tzDQdNqNZ4wZwGOZT_ X6_b8RGUxiUI1cFFOg9rncEOz9bhCwTIG1ITNHn8Odrh8ZQoSCL3hAVmso0eEU_LKNX5qhIkEK6T I9xeu0sP3Iz1n.SJO9wkhqV3guQl.6tCftyrAabbaZ1dToJrAn.67aSevXF2eqy03MIkZn.FmwgC 3FrMuFvr6Yz4K0ANIgEwzM_lcg160DTALEUuyeObT3zxivZOfnJcgRlZ1ipOW_baNAMrds2d5J4h dryrxOxg5lmXb_Q3t_mRkbPQm7CKpl6ie6DHi9neGaxBtXxrEIXkR7SmU3oJKoGNdBRGNW6cXuRn o0phmE2iMFR05E949RFqzgNxQkKH533gLJpXyOook_IGcRrDuil0apNcwbS5g8NPLFI8bnbx_.Oj NGd7oV_Szmrncir2QRHd9xUSaAuLPcOTdOEfFMeAlPVQOtwQh7f8hvmD5v8CC6nucamg9chPsJaL jU9wgvH80ivvgQSVJV1WvkGgRG717_8ndSuPYjfKRifp0gMbaqs63MDz1H5PPG25SRXhS2zYmPon rbUyrFLKgI3XMOlvaTt6c1.0rBtMg8toceg58yO8GLEQv.DyIl7Av03wh5zBCjm3p6BdAXaPTt.W .hQoGMWzf45TBaz0KNnl34qVH3XpjeXnWqmdgXVjZfQw4TIurigSX.sLfAJ9EsMds.becOgXDN9A O9ICOXKbixyDTF_XHrm1EOJwSwvFm7Rgj1CfvQsRoLGQgwIQ_Jm9gfWmfYslKjKrKERLyyMLteDa qt6_3FNPZwUoQqjZZM0ngGDWZOks1tozQJCmyN15PGQWNjFDtk7SOuhs9m80GF1.GsuZK1byiOas 60TEtWNinW6bxYAWkNaAVx1kL2lVXrZqBvwRa79P2uk3A7.I9nUlHZQZRmDuGCfofvV.QwLJkwjF euxjfNomssxN11JocpxqZwKAE51R_YgkzJxuu9e0rp_jLGzNmVUUaMt3saJ1.6TcDtW8bbr.GLt. TlBob.ByCpNdevJBP3eh16ZoiMzZfdff9mUBRvvdXnx8fSz7dE6bGU1ZVQTiScDQxN9iX41nbT46 88eoqaXklE_QmbtDIGRHZLuyBF9cKh_vwI9Cana8- X-Sonic-MF: X-Sonic-ID: e8f3a3cd-1f1e-48bf-9ee9-d5483f2998aa Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Tue, 6 Aug 2024 18:27:34 +0000 Received: by hermes--production-gq1-5d95dc458-sd55t (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID bd1eb19ea6e3a30f65d9732d71d41076; Tue, 06 Aug 2024 18:27:31 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: Any known way to build devel/llvm* ( such as devel/llvm19 ) with --threads=1 for its linker activity during the build? From: Mark Millard In-Reply-To: <0853529b-eb45-43c7-9957-8fe23001c94a@freebsd.org> Date: Tue, 6 Aug 2024 11:27:21 -0700 Cc: Dimitry Andric , FreeBSD Toolchain , FreeBSD ARM List Content-Transfer-Encoding: quoted-printable Message-Id: <0C648446-D075-4981-B348-1525DD1ECD8A@yahoo.com> References: <4FFD603F-E67C-4B62-B91B-8BE365EAA050@yahoo.com> <82E78798-C376-45C4-80FE-96AD14229419@yahoo.com> <0b3b532c-ae94-439c-81aa-9e80a08af43f@freebsd.org> <4b5a6fa1-3b08-4b8b-8577-c548116d2115@freebsd.org> <0853529b-eb45-43c7-9957-8fe23001c94a@freebsd.org> To: mmel@freebsd.org X-Mailer: Apple Mail (2.3774.600.62) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4WdhcS3Jxbz4Dpd On Aug 6, 2024, at 01:56, meloun.michal@gmail.com wrote: > On 05.08.2024 23:35, Mark Millard wrote: >> On Aug 5, 2024, at 08:57, meloun.michal@gmail.com wrote: >>> On 05.08.2024 11:09, Mark Millard wrote: >>>> On Aug 5, 2024, at 00:44, meloun.michal@gmail.com wrote: >>>>>> . . . >>>>> Tegra has 4 Cortex-A15 cores and 2 GB of RAM. >>>> OrangePi+ 2ed: Cortex-A7 with 4 cores and 2 GiBytes of RAM. >>>> I wonder if the 2483 MiBytes would end up being about the >>>> same on the Tegra variation indicated. >>>=20 >>> Yep, it must be +/-same. The 2/2 GB for userland/kernel is defined = by HW. Only size of shared libraries may affect (lower) usable user = space for given program. >>>>> All ports are built with default options. The only non-standard = item is the swap size -> I have 16GB of swap on a swap partition on the = SSD. >>>> Wow, 16 GiBYtes of swap space for 2 GiBytes of RAM. I guess >>>> when the swap is added that you get a notice-pair of the >>>> structure: >>>> QUOTE >>>> warning: total configured swap (. . . pages) exceeds maximum = recommended amount (. . . pages). >>>> warning: increase kern.maxswzone or reduce amount of swap. >>>> END QUOTE >>>> with a rather large difference between the two ". . ." figures. >>>> Do you make other adjustments to deal with the otherwise-reported >>>> potential mistuning? It appears to make tradeoffs in the kernel >>>> internal memory handling, if I understand right. >>> The above message should be interpreted as: warning, the kernel may = in word, rear case need to allocate additional >>> memory when swapping some object(memory) out. This may leads to = deadlock/panic. But again, event is this warning valid, >>> resulting deadlock/panic is very rare. I newer see it in past many = years... >> Interesting. >>>>> But I guess that's not important in this case. >>>> At least for my context, it appears that memory allocations >>>> are failing to find a big enough free area inside the >>>> process's address space --without running out of system >>>> RAM+SWAP space overall. >>>> For the OrangePi+ 2ed ( and devel/llvm18 18.1.7 ) it was >>>> during the earlier linker run for: >>>> FAILED: bin/lli-child-target >>>> . . . >>>> LLVM ERROR: out of memory >>>> Allocation failed >>>> That much finished just fine on the Windows DevKit >>>> 2023 used via a armv7 jail ( devel/llvm18 18.1.8_1 ). >>>> The failure point was in a later link ( matching what >>>> I saw via devel/llvm19 ). >>>>> I just started build of llvm19 - but it takes few hours to = complete.. >>>> Probably fewer hours than on the OrangePi+ 2ed but >>>> more than on the Windows DevKit 2023 (if they were >>>> completing, anyway). >>>=20 >>> The native build is still running (60% in fact), arm32 jail build = has been stopped on my Honeycomb (killed by OOM).Unfortunately this is = an old problem and is common on all platforms. The current LLVM cannot = be built without additional tricks on machines that have less than 2GB = (RAM + swap) per core..... >> FYI: A small RAM+SWAP test for amd64-as-amd64 for devel/llvm18 18.1.7 = . . . >> USE_TMPFS=3Dno >> Targetting BE_NATIVE without BE_AMDGPU and without MLIR. Avoiding = being stripped. >> (MLIR is resource and time expensive to build but I make no use of = it.) >> (I also make no use of BE_AMDGPU materials.) >> (I've not been doing any cross builds in a very long time.) >> (Not stripping can make for better failure reporting.) >> (Leading whitespace might notn b e preserved in the copy of the = diff:) >> # git -C /usr/ports/ diff devel/llvm18 >> diff --git a/devel/llvm18/Makefile b/devel/llvm18/Makefile >> index 1b1f759ba50e..779ddf3bea7e 100644 >> --- a/devel/llvm18/Makefile >> +++ b/devel/llvm18/Makefile >> @@ -87,7 +87,7 @@ CMAKE_ARGS+=3D -DLLVM_ENABLE_TERMINFO=3DOFF >> CMAKE_ARGS+=3D -DLLVM_VERSION_SUFFIX=3D >> OPTIONS_DEFINE=3D BE_AMDGPU BE_WASM CLANG COMPILER_RT DOCS = LLD STATIC_LIBS >> -OPTIONS_DEFAULT=3D BE_AMDGPU BE_WASM CLANG LLD >> +OPTIONS_DEFAULT=3D BE_WASM CLANG LLD >> OPTIONS_SINGLE=3D BACKENDS >> OPTIONS_SINGLE_BACKENDS=3DBE_FREEBSD BE_NATIVE BE_STANDARD >> OPTIONS_EXCLUDE_armv6=3D COMPILER_RT >> @@ -103,7 +103,7 @@ OPTIONS_DEFINE_powerpc=3D GOLD >> OPTIONS_DEFINE_powerpc64=3D GOLD >> OPTIONS_DEFINE_powerpc64le=3D GOLD >> -OPTIONS_DEFAULT+=3D BE_STANDARD COMPILER_RT EXTRAS LIT LLDB = MLIR OPENMP \ >> +OPTIONS_DEFAULT+=3D BE_NATIVE COMPILER_RT EXTRAS LIT LLDB = OPENMP \ >> PYCLANG POLLY STATIC_LIBS >> OPTIONS_DEFAULT_amd64=3D GOLD >> OPTIONS_DEFAULT_powerpc=3D GOLD >> @@ -211,8 +211,8 @@ CONFLICTS_INSTALL=3D ${PORTNAME}${LLVM_SUFFIX} = ${PORTNAME}${LLVM_SUFFIX}-lite >> .if defined(WITH_DEBUG) >> CMAKE_BUILD_TYPE=3D RelWithDebInfo >> -STRIP=3D >> .endif >> +STRIP=3D >> PLIST_SUB+=3D LLVM_MAJOR_MINOR=3D${LLVM_MAJOR_MINOR} \ >> LLVM_MAJOR=3D${LLVM_MAJOR} \ >> @@ -224,10 +224,10 @@ FIRST_COMMAND=3D = ${COMMANDS:C/^/XXXX/1:MXXXX*:C/^XXXX//} >> MAN1SRCS+=3D ${LLVM_MAN1SRCS} >> -STRIP_LIBS=3D BugpointPasses.so \ >> - LLVMHello.so \ >> - ${LIBNAME}.0 \ >> - libLTO.so >> +#STRIP_LIBS=3D BugpointPasses.so \ >> +# LLVMHello.so \ >> +# ${LIBNAME}.0 \ >> +# libLTO.so >> EXTRAS_LIBS=3D \ >> libclangApplyReplacements \ >> Note: This amd64 testing is not using --threads=3D1 for linking. >> A Hyper-V context for amd64-as-amd64 set up for small memory testing = with 4 Hyper-V cores: >> real memory =3D 2147483648 (2048 MB) >> avail memory =3D 2029309952 (1935 MB) >> Swap: 3584Mi Total >> So: AVAIL_RAM+SWAP =3D=3D 5519 MiBytes >> Building just llvm18-18.1.7 : >> [00:00:22] [01] [00:00:00] Building devel/llvm18@default | = llvm18-18.1.7 >> [00:55:27] [01] [00:55:05] Finished devel/llvm18@default | = llvm18-18.1.7: Success ending TMPFS: 0.00 GiB >> (My poudriere-devel is patched to report the ending TMPFS figures.) >> =46rom my patched-up top's output from monitoring during the build: >> Mem: . . . , 1462Mi MaxObsActive, 582432Ki = MaxObsWired, 1940Mi MaxObs(Act+Wir+Lndry) >> Swap: 3584Mi Total , . . . , 1090Mi MaxObsUsed, 2455Mi = MaxObs(Act+Lndry+SwapUsed), 2995Mi MaxObs(A+Wir+L+SU), >> 2996Mi (A+W+L+SU+InAct) >> So, slightly under 3 GiBytes RAM+SWAP observed as used, well under 8 = GiBytes. >> 3GiByte[RAM+SWAP]/4Core =3D=3D 0.75 GiByte[RAM+SWAP]/Core, well under = 2 GiByte[RAM+SWAP]/Core. >> =46rom this and the like of your OOM reporting, I get an idea how = much MLIR, BE_AMDGPU, and BE_STANDARD contribute to the RAM+SWAP use. = (I've not tested such directly in a long time.) >> For reference: >> # uname -apKU >> FreeBSD 7950X3D-UFS 15.0-CURRENT FreeBSD 15.0-CURRENT #144 = main-n271413-1f7df7570174-dirty: Sat Jul 27 16:01:52 UTC 2024 = root@7950X3D-ZFS:/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64= .amd64/sys/GENERIC-NODBG amd64 amd64 1500021 1500021 >> # ~/fbsd-based-on-what-commit.sh -C /usr/main-src >> 1f7df7570174 (HEAD -> main, freebsd/main, freebsd/HEAD) LinuxKPI: = move __kmalloc from slab.h to slab.c >> Author: Bjoern A. Zeeb >> Commit: Bjoern A. Zeeb >> CommitDate: 2024-07-26 11:51:04 +0000 >> branch: main >> merge-base: 1f7df757017404011732196e65981d9325f7a89f >> merge-base: CommitDate: 2024-07-26 11:51:04 +0000 >> n271413 (--first-parent --count for merge-base) >> # ~/fbsd-based-on-what-commit.sh -C /usr/ports >> 4e0616a2066d (HEAD -> main, freebsd/main, freebsd/HEAD) = graphics/libavif: drop maintainership >> Author: Jan Beich >> Commit: Jan Beich >> CommitDate: 2024-07-13 01:31:18 +0000 >> branch: main >> merge-base: 4e0616a2066dc9e45cef47de812efdea4e09709c >> merge-base: CommitDate: 2024-07-13 01:31:18 +0000 >> n671165 (--first-parent --count for merge-base) >> Note: This /usr/ports/ is from before devel/llvm19 or = /devel/freebsd-gcc14 . I've not synchronized this UFS environment with = my experimention with a more recent ports vintage. devel/llvm18 is at = 18.1.7 still here. >> Hyper-V is actually using the same UFS boot media as the native UFS = FreeBSD boot does --but is using a smaller swap partition. >=20 > So, I'm finally able to build llvm19, on native armv7 and also on = arm32 jail. > First It needs the not yet promoted patch >=20 > https://www.mail-archive.com/lldb-commits@lists.llvm.org/msg95950.html Good to know to expect that. Thanks. > But the more serious problem is that llvm's memory requirements are = out of control. > llvm ports with default configuration causes OOM on a 4-core armv7 = with 4G RAM/16G swap, on a 4-core aarch64 with 4G RAM/16G swap and on a = 16-core aarch64 with 16G RAM/16G swap. > It's very hard to name this as correct/expected behavior. (Added dim@ = just FYI). As I understand it, the primary purpose of OPTIONS_DEFAULT is to produce the set of packages intended to be published via the package mirrors that contribute to https://pkg.freebsd.org/ . So, for example, if any such item for a platform is to involve MLIR from a devel/llvm* then that devel/llvm* needs to have MLIR in OPTIONS_DEFAULT for that platform. (MLIR is just an example here, but is a good one.) As far as I can tell, there is no implication that folks building their own packages (or just ports) should build with, for example, MLIR unless a dependency involved requires such. Going the other way, I do not see a reason to require all port options to be "cheap enough" for general use for most everyone that builds packages or ports. So, if the build servers used to populate https://pkg.freebsd.org/ mirrors build what OPTIONS_DEFAULT indicates without OOM activity problems, for example, that is all that is required relative to OPTIONS_DEFAULT and OOM as far as I can tell. This might mean that devel/llvm19 for armv7, for example, needs to not have MLIR in OPTIONS_DEFAULT if the ampere* servers can not build devel/llvm19 for armv7 when MLIR is included. But it would appear that the 64-bit platforms are likely okay with MLIR included as long as there are other packages (or FLANG use) dependent on MLIR that are intended to be supplied as well. Such contributes to why I tailor the OPTIONS_DEFAULT for the devel/llvm* that I use (and, so, build) to avoid the likes of=20 MLIR (that I do not use but has non-trivial consequences if it is built): I do not expect the OPTIONS_DEFAULT's in the ports git repository to happen to be appropriate for my context when I run into some property that does not fit my context. > Mark, I think that root case of your problems is that you're trying to = link with libraries (shared or local) with debug information = (unstriped). Seems likely that more needs to be stripped (or never generated) than before. I'll probably manage to regenerate the armv7 context and test that at some point. It is definitely true that I've historically kept more of such information around than FreeBSD does on its own for optimized code: with symbols or debug information. But it seems that such no longer fits in the armv7 context for what I've historically done. > This causes a huge memory consumption. I don't remember if it just = wastes address space (because mmapping a large library but never = accessing the debugging information) or if it accessing the debugging = information (so the link needs physical memory for it as well). =3D=3D=3D Mark Millard marklmi at yahoo.com