From nobody Thu May 25 04:04:43 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QRZFl34Zmz4CbJ0 for ; Thu, 25 May 2023 04:04:59 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-8.consmr.mail.gq1.yahoo.com (sonic316-8.consmr.mail.gq1.yahoo.com [98.137.69.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4QRZFl0ZmYz46qX for ; Thu, 25 May 2023 04:04:59 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1684987496; bh=czNhpG7LPtnOnoN6SZOdSDexJp/b7GG8ioy5Rg62QiE=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=eGvrIgBiBDj/gU6DUmdtB+96bl23IDG+FMW+rAyy7p1wQtwj9BHPaoXyyEieeMOVtm7atSi4yjWwzObQZyZE6/F+VXsiyrIIQAXRCh1vrX+b2Dwi7PUZy0bjJmrGNCTteb6ntbJRJG8a8py+Ry0XjD58iyvAne2Z2LUV6UYHaKDSkbefHjneTgn6T2lRtlMu062CJAWdlXSEInNblcMfwzJMII07cjQTGwzzBD0BrL/RkSLO9NBEUpaPtaeyBh5KayJ3nzEju36ZV763PUYEhH54oE74NcUwH7+MW7vCUj+iiLa37bByrQJsCqk+rh7xqUydK5I1j+30l44K6KIBgQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1684987496; bh=GN6UeohZXy6KTfeTg3E4t7sGkukc4lzFq/UWIfY0w5k=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=MNdEihxqT0NDnC2Z1Z45UXsE4cH5u1Ujhkam3SVBNrVpW/NzGXkdQR6dMh0/oh3/d6IRF1v1xiivesHSy1Gg6zLig0jbmWOazWDG6zQaax9wACcJUuYpuDTtcwsaCxXXBBCo48Z8PtrX0wVP+vd56n3CDzMdFZsGDcG4VNhDpqieLg/K0AutXT3h4Etkz5rfBdKoQSwY6EoAVV553ZjueCLY5KbyHc4kUoCnU8MytBIx3VhSz1ptXH/CwwRk4OxaieGeBsXxejtL5CDZRAv9Xxzgeq7xtmQ2R5MPg2uvYzelkC+ZHUMnHqT1QsohC6eKq13CAMR9mXz6qR2Jm1MxWA== X-YMail-OSG: ADOY_94VM1ktb.VFBgHCFmUce7TX.bB0CE89VsuzYJ.uYxgYrrcSiEn8julUEXu 5viSNa2.7TNFdzUus_7kUWsiqb2iIBQp.UwIWpsT4OA.WC5_gH1loccowOezURF_77GaJXCY5ecC 8PEF6JLjme_1X2YyWBVVqIymjOfOYCsxlLu1F15POfi06H9Homj_b2zy4wxfLIyezafHnhfXh1t. sBc2kLoORuVW8qU3Y8mtktmeTOIOKHajaN_v7eYH9nsWeNHO0RA9WERjqsDQgH1tgaVOvRXp0MRf GAHQR56Nu01U2muX5R5ZTGkseO2CF04f4FznXon1Nn_cp1guIIGPsyLTWHSHiFtzaoGvPsjOq70O c2zj2kvpFqHRhxtld4F9JHv6veOGtfwh3f09KP8WQpOL1ZINZSMG8hHVMQ91jtohCzxMSRhDWNgm JLp6Taj26BrJ6hstxLBDpFQ7VKhElKFxkETcvn2JY8Jemirz82A5mu7C3IZs9bbTwUeXBUFah_no kSIpHZ3h1zp00_eM1ne3ntKiBbMW8acoF4Ibdqq.Zd1s256CJ3vB7iFV7.s8D7JohjQqp5AF.LZT sn_Qek9tExH6ciCR5pGJIMrQZtZdRUn2puOVI01TaFBu8KufsXIUp_MFGbWpC.rpxmjcSzwUiFmX r9ZF3LxDwiDPnycxnMqFPtoyxtEJZVow2BCpIXALTEq7wB_Y1xTbWf6QQo29IVkcmhgcWIQAxq0w CC7kzzHSMohkX3Dk6OrR7rVppKWuXUzWYqu4m6fWkw1xj9NimPUXy_2Pb3FSmUP0m_OMczDa6I0g O5gOJIqx2lIbhSsCdwtmQfpuqV8yvHpmM7VU1qPaXrUhYEVRFVKmYVeqobihgVFnBv16J0a2uC4J rmeh6m8UiAZ8Xz7MABWn.NX4yaw03dJq6wH.hi60mqcx8jnle28XcmzAN.QWVCBY1HSyBtQ7F6jy iF2df4SVSPb043j1QvaPUNjayUe9AKWTC0mw6eBkEk7J2so0V5Dx522zAtgFV4iqlOmNskMJiu5B pzrR2Y7cqh.GoKoM_9ZBrpwWmvwrQh_SqEDjGqu9Mo6Hbnp7QAUrHJ.ihoxopePe4ZtzYPx7U9cq 6Qal3lNE4e2euXa.JYUshacDUnjZ6XmSA24Khk.qgBywT4MHd63gbbbTID4Flr9xswAmPoNXBlxc LnxdrA9tgnP2eEfsOfkJe7yq4KX0GmgG.nsey_nVJgjAFvEUAMKnpIxEFiQb9YbclnAskvNp5FsP fdGv92XOKY1ZXffvkMYbmn8.GuodaJ0R8fdm4f.M4xwkQp1b1t._wzH36.gV35U1CckNpwfKTYjA pY07ZnFARw_ovlyEeCG81xHVaWr8fLHlRyzOSSdwr7E4Z4UYbAqpDPmnPdEPRaWojtrfOIhQYm8_ _a7i.lsba15epy2BAB6zxBtPGx2UELSgj0z5ozKVlFT_7NFFbg.WbSSq51wsdYxLOu2lylDINpsZ Siz5XAVLyoym_lWFFIlEZMVHuzPcvu7MZR8Es7T833yoJVEQHIwRjlxLK1IXZKpg5TIg19IcXCY2 Hl8JIbpIqtGwZ64lXkaA9z2uUobYB.Yl9CQd8rbZ04.WZHbQGIzpE3Jvr2H.Vkx7mOzu7pr1bPwc 6r7nnWAll4vksmHneTvncRP_StfOHLm93Arzr9_eRq8Kffw1d8TrTwbjPR4b7iFWNf3qHNTDZUzt unuB788Oi._0pGfLAGdeFDktGB3eO3QPlb1X_Hh30RwNOaqxuI7O.cvD5dF2mqfuHcHJeSDfdB.s 9hPa19GGww8QGvcTiuvDupdsQGrJ1etOkmksv5TSvIqR78m8OO4L6xIrWcdlAe1GAMXQ.LTlWMlf Bg9yCill4uwXeOVXtpWOrRi2ETJeiWSBCs69s0.tAnXxUFXPLLVzMIxZurV3MAie0vHup0s7WMCt NYcdErGGGutZnxwp5jjNd2LFU9Y0f5Pnqxpg8.f9i.VJVmBSMPEMvUbr_DsQ1XoHTcCIFSlrhcSA vV1qQs0DpsWZLu3Wu52dk_wWQCzpPmvEidWAulr2JOFJZaPK5hpS32HZQ1zSPGiAHBiNX6B0XUf8 hD1.XZA73wlhDXzxSEj6yRMQkIL6VXlZAt21LPtCVS0ArhmzF4w3u5aJ8C_F9JF9ag.Yt9qrJyNy OegyXmdhNd7HZrHWDIvtvZ4ClQAi5pCAkeujFkJR2QHokrQ9seedWvanwc.bAvtzYREv8.7Il_1Y C1e8Ow1iP3CMizrvOREhhBOdlDYHXm_IgP3doTeXNJN7CidHl89QktlFcjjcwMfuIffM_VLJ8o_w - X-Sonic-MF: X-Sonic-ID: 06748734-f7b6-49ea-8cf7-11ce433d12eb Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Thu, 25 May 2023 04:04:56 +0000 Received: by hermes--production-bf1-54475bbfff-5bcbd (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 17cff9fbf25dd654ed71cd0f50c80032; Thu, 25 May 2023 04:04:55 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: RPi3B -j4 vs. -j3 main [so: 14] from-scratch buildworld times for my context; buildkernel too; swap space usage and such From: Mark Millard In-Reply-To: Date: Wed, 24 May 2023 21:04:43 -0700 Cc: freebsd-arm Content-Transfer-Encoding: quoted-printable Message-Id: <5104F70F-C9C7-4EB6-8764-00C14EF68049@yahoo.com> References: <7758885B-3115-47F0-A453-1C010313D4B8@yahoo.com> <03F330A1-35E4-40D4-B9C6-407041BBEC58@yahoo.com> <7AA87E52-AB3D-48E4-B62A-EE73EBB23003@yahoo.com> <3C1F1156-F0CA-4958-B2D1-A28B2F2D36F5@yahoo.com> To: bob prohaska X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Rspamd-Queue-Id: 4QRZFl0ZmYz46qX X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On May 24, 2023, at 18:51, bob prohaska wrote: > On Wed, May 24, 2023 at 05:46:11PM -0700, Mark Millard wrote: >> RPi3B -j4 vs. -j3 buildworld times for my context: >>=20 >> World built in 120764 seconds, ncpu: 4, make -j4 [So a little under = 33 hr 35 min] >> World built in 115635 seconds, ncpu: 4, make -j3 [So a little under = 32 hr 10 min] >> [A delta of a little under 1hr 30min] >>=20 >> So: -j4 buildworld spent more time waiting for its trashing of >> the swap space than time it gained from having use of a 4th >> core. The trashing is mostly during building of libllvm, libclang, >> and liblldb. The RPi3B RAM subsystem can limit the gain from >> having more cores active as well. >>=20 >>=20 >> By contrast . . . >>=20 >> RPi3B -j4 vs. -j3 buildkernel times for my context: >>=20 >> Kernel(s) GENERIC-NODBG-CA53 built in 7836 seconds, ncpu: 4, make = -j4 [So a little under 2 hr 15 min] >> Kernel(s) GENERIC-NODBG-CA53 built in 8723 seconds, ncpu: 4, make = -j3 [So a little under 2 hr 30 min] >> [A delta of a little under 15 min] >>=20 >> So: -j4 buildkernel spent less time waiting for its trashing of >> the swap space than time it gained from having use of a 4th >> core. (Not much thrashing occurred.) >>=20 >>=20 >> And mem/swap usage info for buildworld+buildkernel . . . >>=20 >> Overall -j4 vs -j3 buildworld buildkernel info for my context: >>=20 >> -j4 Mem: . . ., 677688Ki MaxObsActive, 249652Ki MaxObsWired, 950032Ki = MaxObs(Act+Wir+Lndry) >> -j3 Mem: . . ., 683416Ki MaxObsActive, 315140Ki MaxObsWired, 927424Ki = MaxObs(Act+Wir+Lndry) >>=20 >> -j4 Swap: . . ., 1495Mi MaxObsUsed, 2117Mi = MaxObs(Act+Lndry+SwapUsed), 2358Mi MaxObs(Act+Wir+Lndry+SwapUsed) >> -j3 Swap: . . ., 1178Mi MaxObsUsed, 1811Mi = MaxObs(Act+Lndry+SwapUsed), 2049Mi MaxObs(Act+Wir+Lndry+SwapUsed) >>=20 >>=20 >>=20 >> FYI for the context: >> make[1]: "/usr/main-src/Makefile.inc1" line 326: SYSTEM_COMPILER: = Determined that CC=3Dcc matches the source tree. Not bootstrapping a = cross-compiler. >> make[1]: "/usr/main-src/Makefile.inc1" line 331: SYSTEM_LINKER: = Determined that LD=3Dld matches the source tree. Not bootstrapping a = cross-linker. >>=20 >>=20 >> Notes: >>=20 >> Incremental buildworld's would depend on how much rebuilding of >> libllvm, libclang, and liblldb would happen to occur. >>=20 >> A system with 2 GiBytes of RAM would have far less trashing of >> the swap space. A system with 4 GiBytes of RAM would not thrash >> the swap space. The closest comparison I could make with 4 >> GiBytes of RAM would be the Rock64 doing a from-scratch build. >> It is also cortex-a53 based. As I remember, its RAM subsystem >> does not limit multiple cores as easily/much. I've no access to >> an analogous 2 GiByte context. >>=20 >=20 > Would a DRAM-backed USB "drive" used only for swap help any? I don't > think it's practical, but I'm curious in principle. Long ago I think > folks actually made hardware consisting of dynamic RAM coupled to a > disk interface, probably SCSI, to get around physical RAM limits > on older computers. It's kinda silly for a Pi, and expensive. (Notes presumes well implemented. Notes ignore uses like crash dumps that may need to have data survive power loss.) Compared to spinning rust? Sure, especially for latency contributions to taking time. Compared to the NVMe USB3 drives I'm using? I expect so. But the RPi3B limits the data rate when it is actually transferring data. DRAM would not change that. Compared to a 960 GB U2 Optane used via a USB3 adapter? I'm not sure at what point the USB2 interface ends up being the primary bottleneck, such that detecting differences in various low latency media becomes difficult. But, what capacity? Cost vs an aarch64 with the capacity in its RAM for capacities like 4 GiByte or 8 GiByte or . . . ? Having a 4 GiByte+ aarch64 system is probably more effective. Of course, there are ports that are bigger builds than a FreeBSD buildworld buildkernel. So the problem moves around. But the extra capacity helps all cases. FYI: The -j4 Rock64 buildworld buildkernel is in progress. The Rock64 is set to 1200 MHz for its clock. (The 1296 MHz setting never provided a reliable context for what I have access to.) FYI: I've added a "(sample) mean so far of %idle" to my set of top hacks. Things like thrashing the swap space would drive up the mean compared to not having to wait for the I/O. (Waiting for swap space I/O counts as idle time.) Thus, if the sample mean is much greater than 25% in a 4 core context that is thrashing for -j4, it suggests that trying -j3 might cut the overall time. Similarly for some other %idle figures and -jN numbers. (I start the top after the build starts and use figures from during the build, not a significant time after.) =3D=3D=3D Mark Millard marklmi at yahoo.com