From nobody Sat May 22 23:51:31 2021 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C05DF8F629A for ; Sat, 22 May 2021 23:51:38 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-25.consmr.mail.gq1.yahoo.com (sonic303-25.consmr.mail.gq1.yahoo.com [98.137.64.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FngHF5sYCz3LLw for ; Sat, 22 May 2021 23:51:37 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1621727496; bh=+MHLveNFMCOmwaNw1XB3Ay3wqtauade2uZZo4VYhtMc=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=AK6HPXaXwUR5JlyFCP4FEM9gQ71sDvtCiI8ktI0n9UdobzezKWvAb+XUO/NZOzbfXhkjMENQzr5mBTwuAlbbFXsL2ayLwDGy32K2TmsZJIxpsbcZLMbCJmjD7avN3wMo8I9uk/xkwjaYc5Hte9RLdKtxXOX4GQV+5Ja+EW7uJH5KYqZXxC78lzopG2HrsuZ4+t4rZFPDY40usvnVoLrOBNbOfDA2qhax4kPzfjWjGgaPem5PD64HXQ+/RAlJ8bbxIyHZJl0jai8UqzDxdC5mVk+hNHzYdBCT/EsplxQIRL0n3EAbcb0drth8EQkK+jRjFEsgCPdLckkq1mdGQLh6oQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1621727496; bh=B2G63Oztf0RI01FzCS4fZiXJOAdQZYOJrCD0q1Wc2x/=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=YCDDZgJepLvDRnYoyeaYNL5hBQ7WrvQcOnxMRH8Gfx/qSV1uV0DuDILzV78PjKFB6yUGDOwPmCl+Secjo21LhjVUiD/aLYyuvT7ERHrr/YRebvdt1UA//EKGWEJA14/Gz+TL2V6lb09zc8yOxEvpoUI+bQBCVqjFvG66a5I4O0vSwOopwzg3K6Ub0FmGtsEJ1xrQiSzEjUsLEMf/GOohJR9VF8Q5VFj150IDT+ZmJUQBOX8M6V3IbuvCEKTru+3TQ+PuG8qLeXQrsAWGhWKlUpBwyEv37cvlx3OCKJiFDLzsdZQexS5/Ob31Nt1UFWvz7mODlnqzXV8qU6qrfKFUsw== X-YMail-OSG: e8MpwAoVM1kdJqOtHkHcN7Opg5jSLKrs0w2Ha.7RMqVb9nPC5U9EIMoMi9IuqTj pC2QjWwjnrS9cFhbmAkshsTK89UZ2n6ulwCZW2Jz_UDroFvXpPeYqcOYJs0l.HLkiPBMBkB4TiiS 7pgJgkT6yLsYjyNNN8W1p9mg4VxrC_S6mRG5iVNJSYPtZUU47Ye45Rwt0K_6n7Ng8F4K0zVCz8.W Xl7syCk_.HqLHL8I9sP.reK6zDcI91e5PZ7.Hm441DFcu_4izdwXDEJwcVcMYAR1FFF2ER17N1vm VN7kfo8LAPXD4D7JZASbblhNcw9yAKXJnOrQerX06opPqPgGo44ouD2TOaQ2UEs4JyU7OTYsvDrs 3RJmrRE59vqANJ_MHVl4u3rQ1APoc4941SO4OgH_3A2wM9DMbyf_dP5k_62PfJtpDlKCNH0v0HjP u6ljXskkNj6uVC97taDnGV13Fa2rd3lSBVonFZlF297sE3h4Yg.Pv3lo4bJjg._ouJHpsq3Vi181 iPuR16tAJ9wvMjJt82hCctAP_9lc9mC0jOVp1OCLN79SVZsM7Pmj8N3tdSyUQ2TXcD_0M2B83GJt CDl1v2xrJ_GGQu3uMlhp7R4JzV4WTQwD9bf7dBRykFxBIKL0EwlMlSXZtJX3TpP.l5kDYBcd37Wp w_zb0enOWBKaLU9iOXVWvsSg0VbnUYOefiew4jglpbGUXiLvHD6zCqT8rJmASUiim2PUhkFsoUaj _S4_iRsBCMsXKvIWVNaenF8SLav8ZgLhBxGS0PMOPVpp2HGsKDrtZlPKCrLIbDXrLk8DM7488CIJ 6U3uyfpSgBJ7KSScpWW4vLY1eL8giVd1RYwLVamBjvvrAC9Ze9TiqUzB.ehMZlC5s5jHlHBgIjGg eRxadQoLDj9RErbZqC9ekPLEuciAMq5cy_7PrGV1RNwgxIElvxJnShFBg6BL7P4SzYHjxVx4m8Ma fmWsbi0a_RzDvIsHsN2tT_gQtArxLuZRyANpjtvR7VhwKPmVZosoEBQc2Vz.KBJWtO.ii31SLjSb AhYkqGGh5yQXFQcqAzNs.56vSg7NNo_nxbcSVihHtn7cN_Enq6ZwTp4qOHOVIzELFhbaUZT.cdkk n1C77cUWccDbnFnyQV3ffnA_6GNF4tM0BXi3DTqCRvCuISfnFZQ2.cBjdAMGS4Z.ol58ndaimPZg xMmGswxBcvSKnBGjD93uztaCauB3Rl1rVWUF3Jn3Vbuh5MmFRoLBWh4A_9OYk15X6NQXerF0sMcz 3wZAtMZ86NgYhVSWqceU5l163raaRx7Ux.G4ummOiE4P33Gsv4pqau.94SJgsPoDAguZvwH3..l. Zes7KBUZg7HvXQEMJQ2T1FjbMY.YN1Js4E9.emobVzvR850ctJwq50orisaiwfBJfceQFtn7dy70 2qpFCyfaUgT5Kjz19RWw_bN9rihG6jcKfzWF8XzZp2ghz202_tkhXyydlwkU7JTH13rMSjPk6ng7 yrH9vKLNCdWj3RC25Mw5f_K.RIf4oRjh3.rBmHn1jZu0lTp_CwXGSb5OSU02TZJxj5Ew8k6R.nwh WrIqPXE3c4p3MwhFUYpPYodryJFVVQgJhuBZZDZGTEuOg5QUFkqk_ENedyCt.iftNmQGCYF_r7AH 5utt9KnbyveJubt0QibK60gknobjJoAIkiJ5dlXC9gdEybEyzNMWjVccRU.KEbyAmgc7hSrnDD_I Sg1DLn7rMg1_W3MAb9IWwM0pfiFJ3w9gOlq3LA3foX0mRSdN2ZpIRs7gzFD2ePkv2d2cL7m1GKi8 Ee4srOo2NFEqNS.e4mPKJvyswYP7kPopIQp4CMzvJaKdrGQafNSB2axkb4e8kOwz8oJUcO6f.xm1 QWW3SXRaurz8mouaSVMJIb5hRMfMGOs7xKfPGuc46rekvoW1NDdCIxv3i6iqqconppGrXQR.ThHa Z2can1UP9njbJ8V7rgLc6zjKboflAjB5ki4N3OSrTdH1RQOD0g48XkH1V_TJidwSOrqZ2gg.BB1a Ap4fOezynSwyJH5F9q_CnFGJ7P9HELpDVL53aIwvJkluXkxfgBczPaptRJagh8HJN1T.oNZSX.dZ henWjj6fx94mjV4aD41neLCKpjixIrIusQNwUkqeqR48cDauHAFFjnzn3NBY4yvkpFi03tJvKAxD 2ztuNuzNG_oeyTDa3ayYfwWHro0nIeKIfYvSY3P9_aIggUzJ5b_X4mPvvRJVnA0z4NLTMA3ltMCq g6g9DYsUfuOWfxIARlk3VFZ2p68XYKNbklD1KeoquXTUS5IYojCcgIDXDZPkdtbqu6PfWme1skG6 6BUd35l1dfgATaSYoWrZiRP7NQQg8HbotaLxkGlMEkevxDpoBUrEqU7Iv3hUerBWwefZ_tG6knw6 m9MrwHnFo4OjVO5SnYRHRObrxW1BrO3UV.RzUxZbudKklvUt4N7pnpXzjosqD1D_KSuPOLryDh6H 9jqiQOO7CvXyt3Rl.VsyDk.XzTHowSfaPWlCEiFNVf2Sgn0pHV7vc2TcwF7dVK48XLChr X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Sat, 22 May 2021 23:51:36 +0000 Received: by kubenode502.mail-prod1.omega.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID e7c8dc3bdf483806204b6c6af9e0e23c; Sat, 22 May 2021 23:51:33 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.80.0.2.43\)) Subject: Re: RPi 4 build time In-Reply-To: Date: Sat, 22 May 2021 16:51:31 -0700 Cc: freebsd-arm@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <9A949E36-FDF2-40B3-A126-5538E41964D3@yahoo.com> References: <0299DFBF-5497-4A06-978D-13E4FBD8B5F0@yahoo.com> To: tech-lists X-Mailer: Apple Mail (2.3654.80.0.2.43) X-Rspamd-Queue-Id: 4FngHF5sYCz3LLw X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=AK6HPXaX; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.206 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-3.50 / 15.00]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.64.206:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.64.206:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.206:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.206:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] Reply-To: marklmi@yahoo.com From: Mark Millard via freebsd-arm X-Original-From: Mark Millard On 2021-May-22, at 13:12, Mark Millard wrote: > On 2021-May-22, at 09:01, tech-lists wrote: >=20 >> On Fri, May 21, 2021 at 03:51:35PM -0700, Mark Millard via = freebsd-arm wrote: >>=20 >>> So, if I read this right, you are reporting 4.5 hrs >>> for a "hot ccache" result, which I had mentioned as >>> one of the things leading to large variations in >>> reported build times. >>=20 >> Hi, >>=20 >> not sure what you mean by "hot cache" >=20 > The first time ccache is used it has no prior results > to use to avoid compiles/links: an empty cache (a form > of "cold" cache). Another form of "cold" cache could > result from changing compiler options that would change > the code generated for (nearly) every file produced so > that the cache becomes ineffective. >=20 > "hot" refers to having a significant amount of > "effective/used cache content" that makes a notable > difference in the build times. I'm not that impressed > with the terminology but it is was I've seen used the > most frequently for ccache. So I used it. >=20 >> - I always use devel/ccache-static >> as have tended to build from source throughout my time of using = freebsd. >> It provides tremendous speedups and generally i'll disable it only if = a >> problem arises and am debugging it, or crossing a version boundary = like >> from stable to current. What I'm saying is I don't know when ccache = was >> last used for building anything. >=20 > I'm confused how you can know it "provides tremendous > speedups" while simultaneously not knowing "when ccache > was last used for building anything". It sounds like you > think the 4.5 hr build might have not have been from > having a notable speed up from ccache? >=20 > Remember that when comparing to my "from scratch" > build times: in my build everything was compiled > and linked, no prior build materials around to be > reused. So I'm reporting a context where I know > how to interpret the result and I'm presenting > enough history to establish a repeatable context. >=20 >> 1. rpi4 here is clocked to 2.0GHz >> 2. ccache is in use and /var/cache/ccache has *not* been previously = cleared >> (i'll clear it for next test) >>=20 >> 3. make cleanworld cleandir clean has been run on /usr/src >> 4. sources are at 246839 >>=20 >> 5. this rpi4 has the following properties for its disk: >> [i] root-on-zfs >> [ii] boot-to-usb3 >> [iii] 4k sectorsize forced >> [iv] encrypted swapspace >> [v] entire filesystem encryption >=20 > FYI: My build-experiment boot media are never > encrypted for the file system or swap/paging > space. Another thing I'd not thought to comment > on. As I've reported, my UFS based and ZFS based > experiments get only minor variations in > build times (variations of minutes for from- > scratch builds that take hours). >=20 >> /etc/src.conf is >> https://cloud.zyxst.net/~john/FreeBSD/rpi4-main/src.conf >>=20 >> make -j10 cleanworld started on Sat May 22 15:41:58 BST 2021 >> make -j10 cleanworld completed on Sat May 22 15:43:23 BST 2021 >>=20 >> make -j10 cleandir started on Sat May 22 15:43:23 BST 2021 >> make -j10 cleandir completed on Sat May 22 15:43:50 BST 2021 >>=20 >> make -j10 clean started on Sat May 22 15:43:50 BST 2021 >> make -j10 clean completed on Sat May 22 15:44:11 BST 2021 >>=20 >> make -j6 buildworld started on Sat May 22 15:44:11 BST 2021 >> make -j6 buildworld completed on Sat May 22 16:20:48 BST 2021 >=20 > So between 36 min and 37 min to rebuild the same version > with the same build options and compiler/link command lines > (near[?] maximal effective-ccache content that leads to > near[?] maximal avoidance of rebuild activity). >=20 > Cool. >=20 > For META_MODE builds, seeing how long it takes to go through > and discover that little or nothing needs to be rebuild would > be the build times for 2nd build from doing back-to-back builds > (not even an install to the live system between). The META_MODE > use would then prevent most rebuild activity. I've not done > such a timing in a long time and it does not approximate any > normal build time for my typical rebuild patterns. So I do not > normally time that. >=20 > I'm not claiming META_MODE is similarly effective to ccache. > In fact, I know of issues where META_MODE rebuilds files that > ccache would avoid rebuilding the same file: for example, > doing an install of a build to the live system between the > rebuilds has side effects that lead META_MODE to rebuild far > more things. >=20 >> make -j6 buildkernel started on Sat May 22 16:20:48 BST 2021 >> make -j6 buildkernel completed on Sat May 22 16:49:18 BST 2021 >=20 > So between 28 min and 29 min to rebuild the same version with > the same build options and compiler/link command lines > (near[?] maximal effective-ccache content). >=20 > Total between 64 min and 66 min overall for buildworld buildkernel > for the near[?] maximal effective-ccache content and needing all > the files. >=20 > Good to know. Thanks. >=20 I happen to have ended up with an opportunity to do (no cleanout of old results after the first rebuild, no installationa of any of the builds): rebuild world reboot rebuild world The 2nd rebuild of world got: World built in 354 seconds, ncpu: 4, make -j4 So a little under 6 minutes via META_MODE. META_MODE does end up causing some rebuild activity, just not much. Much of it is re-linking. I did another "rebuild world" without a new reboot and got: World built in 293 seconds, ncpu: 4, make -j4 So, somewhat under 5 minutes for more context cached in RAM. A similar sequence for a debug build instead of non-debug build (building machine running non-debug) got: World built in 526 seconds, ncpu: 4, make -j4 So, somewhat under 9 minutes. Then (no reboot between): World built in 296 seconds, ncpu: 4, make -j4 So, somewhat under 5 minutes again. In general these figures are approximations of the low bound on a buildworld that is a (near) no-op but is not frequently approached in my normal activity. But it is rare for me to update the source tree again and rebuild after only a few source commits after what was originally rebuilt. For such, sub-half hour rebuilds can certainly occur via META_MODE use. The context happened to be the ZFS based one in all cases. Still no ccache use. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)