From nobody Sun May 23 01:29:03 2021 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 450B18F2842 for ; Sun, 23 May 2021 01:29:10 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic313-21.consmr.mail.gq1.yahoo.com (sonic313-21.consmr.mail.gq1.yahoo.com [98.137.65.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FnjRn6Ymkz4slS for ; Sun, 23 May 2021 01:29:09 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1621733348; bh=LDsGSQIIODYQOFWEHbcFOGdvewn3VYRv+puHNQJ8RZs=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=m8lW6X50oN3JBFXGNLxNfpDlAZSo5TbDPQizsJslvZWC19nphSORR+NjKDamFlDKWd1VjGrdSbx10zoq5LTRbI1/ARssjKXOdvYXwXM7R57HBYcTDutmrzd1LPrgqk+FijCu/06K7FEgAAFiQgNjJYX8x2uEZZzSNkdr/ZjBSGMcNwBnN7kDA5ULx0i3GVfIzMLdRgK3yNvgsOU0UKjcRssvJb5QlmG1qTm/bJk9hjUChRg7yODq7oNxayMlZCwO3gStDzIEClz9mpngI+9DcTRjb5VDSmehCf9DF2IYCkQCWsrizACHihprcEys+a9QDeTvRyS1qU4xtQBHU3X7Kg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1621733348; bh=vsD1ZFnRrEajCV8yRXYDNg5QqvlfQHo2e2eK0jXZRR0=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=CXJ2g1FSW+gG5yKHIWkfS2j6Sd2mzX98T1c493KGUYN1S6w5e/BLvSX6j3z20p5GnfNKverZ1Dd95I/4rk57RfDN+limh3hSe8XtArtt0h/7ibqIMDZ+PTdPQuD1kvn9wBNmbXgLrB/p6Z5AHu6aLkQpJIyzk7l6YpEt2dfXy+ncU9w0ySzl+C+EbK+wxE5NMKda0aR0OwZLccMcFrVUBkaZrTc4LbJ2audeS4c9jED2gy0ZQFXJBpmUBVJZ51vUtWgECAG055rPyUCRS441gnoKwGLD+YL3Mwvg1QSjrZcdevuoox3cMbFwI+75TxOhuLA6eNvgwDZJzPNCNm14uA== X-YMail-OSG: 0olL1NcVM1mSSQi.Nnvb99qkZnl_LcbKHt89Ser3CD8VqPCoP.eU0Q9SoZmfc67 Z5M0EU_cODcdV0p2BY.9ILjhMozWMLkJfMnys4HhIH.rjKcfWfK1F0qnyM.iLTiqs92EDjHqGjwO r6qfeWhlXa1Rew.9PRHAC4mkzSRMffnVyMtuYERgvqypJ_S8YIXPs6dMelsXq4txg9l1SxbxLpwI 84xQQ85rwyf55GT7x2biaQj3gFQ3eQdEIjiI.XQnlnvmocJ0UQHoQocZuzgb_xClyTKyxKRH2s45 LY3OBbI5j4zR6bVw.iLpwgN8EIp3T3ZgvjQNp0F6Ej396wnFxA.lFIYTTBsTj0e8X.jFt0gd4ulj .KrNUbOB95fK3mboxwUCZiziHsY3EPZo5CGaRADBkYvRk6RtIC3UjRY1YF_GiOcVI2fcD5rNBJmX B2O3zij8LELlEBjbJ0yXv6wdPh8pYTXFtlzKULh8svTXoWWf5ILkrlZ53QKmrLsRtVSPyQ1ueCJr Fodw7cSWkgL.Q8LhccNliaPUjA0rCtOBeJilFCvgexz_w62jaS.oZB3Fn1FTUr16DIr5WNMhq5iH op0ZnGrYhM134qCXTsgPTCEjPc8znnEik3sqAgy0_S7DJcARmyzmRMFh7Rcbh.vChx9T.SoUziUi WDuoJVySYijo2DB11J4jQP2gb96UVyqcrWdifGk1ThqEhnpAwvhiu24UuyLj3MazHUDwbCUi4wzM XVBnjVL_pmpRfRCj3ji41EgYPyQwwfaJednC3.K3Q8cZh53PZcr2yTaCQLM4acBFNm4G7f45eSQ6 ia2.OvOEVsY6VqivXEUlrc9PDxj5.nXHzeAb3SQGvbL6yMUas.b0StFqRopvbsdLZReyGEDjNiFQ lN7E5Grv1uOXdm2nByQZX0awWal_F2c0pP6iMX3.x3f2QvZoz12hQFZjbnrrOmWp3WDwbBPXbI2N JjL.P4izKyrE6l6tF0adjz5C1H06BBxkXoHVboKeSC8rzMyEt08YRQ4rMAKkH7jphw_zEn5LDaeR o_b2zEzOcQU5ym.zaOZb5pI63R0z21_l_5_ZBlgHvbpmRxYXxJxhK9b1tig_Hu8f0J50ybWmGZhs GFlDOM3fa4nbnNkXHmSzB3BPPnbnYN7LLtzszOzJ2u.QifOees27uMGNfSK7PZ.Z0znrxzwoDHKC vkcwxxPxGoBIEoM3bafazXo7tPd1SZW3waL.NSZCZmFfbYMj_dlgmrUenxQ0xZqsA462tidX9d2E kJNYjQeM9Eh9m7FqQlrYqfr4cV9Y_A.2hnD7d1tn0xITiyITeldqjUN.xlUh5jcrgAlzG8tFDOCL _E8MBq6oQGMrvi6uKeMHN7NIbS38wHsQAwrbpwAenOXyF1Ucfo_Dz_bpnuE.tZGeoplBlga1rJuL KmVMzvQSp1Vl6QcCdy_cD57Gg77oHTe4xfcprTL4oLfjO9Du49IdLwyIaJ0Igod0e0okB6zdHFzC MRhtVyaHIlJMT7NMQbg5zutvFe3DtSNF8gHdRF3_h3VyktODvjAlNxGSZnYQZYbAzisG36p4fDLw _X0nYQmwLf4eBjWTjVaVFe8KQHc2amp4ct5G78Wqs2MesrESB2j_hLYDcy5A1yxFmVzI8rGCADb1 dMSyc.7_fBI_lAqz8bglpRdTLp7tfynQlFdss32sx3izrfqq332pXXGIj.AYtOAgxA62sd9jJRCO Ih0XKks1Y9dvXYHY6FvZoDGakLOj1eBDb3MYLyTsYJZlmn3SPu.mIr_1opzBpx3.6rpREfFE5rCu dOigqkSqOzlxyKwqFBL8JrI5mlvyc98W_6Wr1E4IkQJN3DZYu_3QWP1fCOT0dyiixVAgZ72neBtW uG6aF.WTvBZNPF4mI0TPj4KEkSoS_0VKGKQBpTFKgjqcw3O98ctBEuq8_hOVfbbaxe.Yiu2XemEJ oo22Sa0B4QsheD6CvGHm7bmJ0lGFdgYBspvJtgxgGvEExW6O7tJaEYixhEpNuqU3Ngmi7t._5KuE Ncmg5rqlglnE9zy87n6a71jOXnYVqv3FOnaMpVs38DuJEe4pHVkCRUB1272fVAyTtPH1pWShVcYb cWfyc5IukajPWvzLmDgjsXvpJwariYZS85PfyQ3Bes9XGqKgepmeZM_Pn6Pe5.FPMCq1r6.Ev.vI Yz4rq81X1pnXIgLeqE.fG29i.lkg_x1Uc.B5w_Ls9WlhodY2EAVMmCeDzKll.aQyTKxPJZIsQuZl ahqrFlko3YCty9m2O76BZuXEyAhKq.O0cjQye0UK_mdbY2wBxqr6JANLv1agjkXEFk8rJOxtJPaM eky1SX8n.8vB.DG3Rr5gCJxdKzfxDMQrGm.g0DxEMMY_Si.AmQd3qITWGNDfMCPe.5tycpRpZ.Oq gpK_MvZ_.QqVj_uLhOHskCsNMc02hCodDibIO20MqTGAF8t3dtlItkBwfBXbbyOOkyTZSmsA63AR YLcZAi2XGCWik X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic313.consmr.mail.gq1.yahoo.com with HTTP; Sun, 23 May 2021 01:29:08 +0000 Received: by kubenode527.mail-prod1.omega.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 312327ac9f8620bc9b9aac902ce1ead2; Sun, 23 May 2021 01:29:03 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.80.0.2.43\)) Subject: Re: RPi 4 build time In-Reply-To: Date: Sat, 22 May 2021 18:29:03 -0700 Cc: freebsd-arm@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <7F7918BB-CDA4-433C-87DD-99EC4C7C68BA@yahoo.com> References: <0299DFBF-5497-4A06-978D-13E4FBD8B5F0@yahoo.com> To: tech-lists X-Mailer: Apple Mail (2.3654.80.0.2.43) X-Rspamd-Queue-Id: 4FnjRn6Ymkz4slS X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Reply-To: marklmi@yahoo.com From: Mark Millard via freebsd-arm X-Original-From: Mark Millard On 2021-May-22, at 18:12, tech-lists wrote: > On Sat, May 22, 2021 at 01:12:23PM -0700, Mark Millard via freebsd-arm = wrote: >> of "cold" cache). Another form of "cold" cache could >> result from changing compiler options that would change >> the code generated for (nearly) every file produced so >> that the cache becomes ineffective. >>=20 >> "hot" refers to having a significant amount of >> "effective/used cache content" that makes a notable >> difference in the build times. I'm not that impressed >> with the terminology but it is was I've seen used the >> most frequently for ccache. So I used it. >=20 > OK >=20 >> I'm confused how you can know it "provides tremendous >> speedups" while simultaneously not knowing "when ccache >> was last used for building anything".=20 >=20 > what I meant was "I'm not sure of the last time I built > anything that used ccache" or, more accurately, "I can't remember the > last time I built anything on that machine" because some building uses > ccache, others not. But I know that buildworld and friends use ccache. >=20 >> Remember that when comparing to my "from scratch" >> build times: in my build everything was compiled >> and linked, no prior build materials around to be >> reused. So I'm reporting a context where I know >> how to interpret the result and I'm presenting >> enough history to establish a repeatable context. >=20 > OK I ran another build. Same sources. >=20 > 1. rm -rf /usr/obj && mkdir /usr/obj > 2. rm -rf /var/cache/ccache && mkdir /var/cache/ccache >=20 > then: >=20 > make -j10 cleanworld started on Sat May 22 19:10:02 BST 2021 > make -j10 cleanworld completed on Sat May 22 19:10:02 BST 2021 > # > make -j10 cleandir started on Sat May 22 19:10:02 BST 2021 > make -j10 cleandir completed on Sat May 22 19:10:37 BST 2021 > # > make -j10 clean started on Sat May 22 19:10:37 BST 2021 > make -j10 clean completed on Sat May 22 19:10:58 BST 2021 > # > make -j6 buildworld started on Sat May 22 19:10:58 BST 2021 > make -j6 buildworld completed on Sun May 23 00:47:03 BST 2021 > # > make -j6 buildkernel started on Sun May 23 00:47:03 BST 2021 > make -j6 buildkernel completed on Sun May 23 01:20:31 BST 2021 >=20 > so buildworld took 5hr 36min 5s and buildkernel 33min 28s from cold. So, in your kind of context, if it is sigifnicantly faster than those figures, you can infer that buildworld and/or buildkernel was using cache. (Presumes you are not also using META_MODE or other such. Otherwise there would be multiple possibilities for sources of avoiding some of the rebuild activity.) > earlier, I wrote: >=20 >>> make -j6 buildworld started on Sat May 22 15:44:11 BST 2021 >>> make -j6 buildworld completed on Sat May 22 16:20:48 BST 2021 >=20 >>> make -j6 buildkernel started on Sat May 22 16:20:48 BST 2021 >>> make -j6 buildkernel completed on Sat May 22 16:49:18 BST 2021 >=20 > 36min 37s for make buildworld and 28min 30s for make buildkernel. This > is what I meant by "tremendous speedups". Other things get built on = this > machine; it has a poudriere instance. So I guess anything using C or = C++ > will use the ccache when building. I might not know exactly *when* > unless I also know (when whatever the machine was compiling), it used > something that ccache could be used for. Nice to have examples of both numbers. Thanks. poudriere has /usr/local/etc/poudriere.conf.sample which contains material about configuring poudreire for ccache use: # ccache support. Supply the path to your ccache cache directory. # It will be mounted into the jail and be shared among all jails. # It is recommended that extra ccache configuration be done with # ccache -o rather than from the environment. #CCACHE_DIR=3D/var/cache/ccache # Static ccache support from host. This uses the existing # ccache from the host in the build jail. This is useful for # using ccache+memcached which cannot easily be bootstrapped # otherwise. The path to the PREFIX where ccache was installed # must be used here, and ccache must have been built statically. # Note also that ccache+memcached will require network access # which is normally disabled. Separately setting RESTRICT_NETWORKING=3Dno= # may be required for non-localhost memcached servers. #CCACHE_STATIC_PREFIX=3D/usr/local and: # List of packages that will always be allowed to use MAKE_JOBS # regardless of ALLOW_MAKE_JOBS. This is useful for allowing ports # which holdup the rest of the queue to build more quickly. #ALLOW_MAKE_JOBS_PACKAGES=3D"pkg ccache py*" and: # Define to yes to build and stage as a regular user # Default: yes, unless CCACHE_DIR is set and CCACHE_DIR_NON_ROOT_SAFE is = not # set. Note that to use ccache with BUILD_AS_NON_ROOT you will need to # use a non-shared CCACHE_DIR that is only built by PORTBUILD_USER and = chowned # to that user. Then set CCACHE_DIR_NON_ROOT_SAFE to yes. #BUILD_AS_NON_ROOT=3Dno and: # A list of directories to exclude from leftover and filesystem = violation # mtree checks. Ccache is used here as an example but is already # excluded by default. There is no need to add it here unless a # special configuration is used where it is a problem. # Default: none #LOCAL_MTREE_EXCLUDES=3D"/usr/obj /var/tmp/ccache" (Not that I've used such.) If I read that right, ccache is not automatically used just because cache is installed. Instead /usr/local/etc/poudriere.conf needs to be adjusted. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)