From nobody Tue Jun 15 10:02:03 2021 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4BAD511D2B99 for ; Tue, 15 Jun 2021 10:02:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-23.consmr.mail.gq1.yahoo.com (sonic304-23.consmr.mail.gq1.yahoo.com [98.137.68.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4G43l81nXHz3GFT for ; Tue, 15 Jun 2021 10:02:11 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1623751330; bh=OiXuFLPnsaEhMQ/dcUYcEwiag5/XWHbZGEJmNMHjYjc=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=HunGKj8fwAfxQtFp8uzNmX1Z5StlJCyturfYTjQaWcNcsPHkTLSvifkBNQpKvbmlHgSIKgHN0rc0K7f4VgZDI945s239FZ2XZ6VxduMyR8H21sZbN1J9IerDJK4J36fRnsFTZZ1jZP12NOgeGJcqgX+quL8OoJkxDixzStfUfw9db6E8TkzRTJT58Ep1D045cbku0MJc6CRZaYOBGw3pJipO660BHWz4BpJPyLPt67xRhWf0K6rOKfDf01oL7yyZmV1nf9aI+Zqoun7IgpVv+pBApDpyeL0MmKMthMEs/O9gcfiiW7RYD5SL9UBJHgmq/JlhkTn76CUU1sGTeI6yBQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1623751330; bh=IM+zVfPobZ6hKzFR1rARLbLQ0aLy8yU33N/mSlwfwTP=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=i8nHV/W6O6dLoABcfOn2mauqrAHOw2xcfoohvAcOrD+GyKVweQftd7o50Wjh7egtRaVUlaNZcYU+NvoQNTGdk5jBVT4rJbfoLwMkbm+zb+S57Q0BldOcsd9P73j4vVkBIPylDfnaEeQf8rhc1CZOU+j1SVnfoxQH0AcKUg4oPxV2BW8Cg1KXibgn1INIfSd8RzkHSVK5a3KAgMXnQJJ43JeJHP/Mboro7dQ5NnPWXDMvAhVg0wf9I9W02l/SU9i0Ycc9UOmABEwQOh05hAPz8Ba5WF/Hw2zVjkaS7C0DkFGPY1J/jbUvlQU/gFY5AtBgr8i0AfP64+1RGhOmMQ9Bhw== X-YMail-OSG: mtRtnOUVM1m2zl4wYi7ugnXklRkgKx0Jsedy8LIeQZ.MWWDvpbOXZFYh9jlMoKg nieSy.DOUjLwIyzcbvewzwCdYaZvuIGsuOhfQ8REIHHvlAB3CDsbRIzP6GE2rR4qPHXqUu_M3iWB Ae5SApiB4MYkY_RdlIPndlhalQYW4lPpnS1aZ_PIt8.Ir.6JtoMjP1qMGE_yud340B2gTA6WGoQY jOlECyhr86PT79.A6L_OMioutU6bvkb1asI8vnlvKV1u0k1MXwM9mQciCYY5.7CTwxXsj8BDx_YC 62uww0_ejLWDOe6QOEC8EkCoJ0ENcduw7s4Mg3WlGS7X7y49x54WxXL1o0ZVVzelbleWE5_lbole sWN7Eq46j9eppq_29HLxe67jZeJ.NSTmW70o1VRk6ldB86ztc_uvIdUUBvNVr4lVU6unN.rYVwlZ 5o5NIGWrFzR_kJn7mBOAiwOqayj703XJ.xqnKRwF5Rh_gqq111z1HTrT0pYM4dmoMZ08srDxw76X kZ1pfxCll3ih7oQG8DwDV7REQf7G4XxH6EZqWrIlpE5nSodFcQNOqj4SJ8AHN2PRMdzYmQOtY8HM 30Zj6L7OIXfGEP84V9kPpBcFCOWuJPJXS2jc8zgXeVHGioE1gD6oVC2PGsLeji2Zc2o7_8hWylyC jUsfIzyTLxO66Pt5oaCNz8xX.6kL8OD5C9.O7miWSK22Mbs_ZrmCBQ.be.l_oPuPNn1JgIUhGHux MmQ_MNub2jreth9rB6RQk_X283450HPQwOGOnnenuXEN3nBulPs2SlAKp2vwRpn7iVsDSVHPZUUi 3U8I8vJCwce7gmqJF92Cg030iy2NJrtSFEc09vypD0yISunPbzgWqp_p7QwJb9bMoS_1VlkGwJs6 HL1kA9A5ErWV5WjRncjmF6FzR4xZjJeDbqapv.4eWIoY8Wnt.xj8leuswkZmkmtvD8mJFDyGPWiK CukK5fBtIALB3rIoOXYjlSdlWHqj5jWiWvr36DTS_ZxG4GGZGPZIqyPg5zLk1OZpbPAY8sjGfclX 2sV_Q88LILdDX4LbrerG_DC1.JNI9LKek0NwFtd5OBwmW4uHcC_Edr54Y99imjExg5nHSz0RysP4 LiaETLdh0uKVap_05.My3WeNl.cv5GPIBIOsOxBt2OLBqy0j1R3wO0pZzcDYXz2GnpEgGFwZPkFr ECEeIfO52K9KkqNwEiOefpKNQ8z3UZCxAtnIa6NNPPDnHH78QJWLCNudRJut_P03dDhNbw2u_03e rrs_r3BPCFfYJb5bs09lKlsAr52bkeAick_jJmxaFJKOKzRk6b7ppa9E1VhIhcD1Y1Gdc8f0Eut2 bGaBRqA171bFIG.k9M0ySZhtOccu6il1uUJjZrjm2GCiwFogT37vF9jYFVhdxK5lnw5SUnwigcgm GLhFvdEZNTcMOYaYYStZE70WnJjqEZUCTvaLUiTx21ILjDzECIc0klbTxcw8A5RjFFvCE4mz15qz _FX72NItERifvchHsciWxOlg_jh7R3IaIP1wyF6Aws2GvW2exMNcO0ao079FWgvGuOi8d6fUkzrn x1R0um0lh_F_uxP8wNGWuHnMsklXJ5z7A8rYFxjJRci0mbHjSWwm3aJxXAQ38OAV.ZrqACRdM9Uo h9shavbkom8ZWyYMprJGkQWS_t_R6cvQnNK94KhGSjtoKr2lbkRrcgkFDHOj4NVQ0rQJj4liUqi6 QckLZYbDOUNJXbiv8N2QNB2BqVubZTd3AE0gMM_gjDcJw9mZxBFzfH0DcJ__3FsoT_dFQ31Q7azZ V5VVtxQ4GVlPeqkhHPbl0a4lq1Gfc_9auQdEAA7CSmBlKmW7.7.MW_LKdTyEN9QekpEB89XNj3Fo fn6.Id03NZ.l5JOfSePwcUXnNwMk8qvYoEgYVJNJlWbZaSs7SyL6MMucs.Xkz_nsWMKhx2qTFPcu OSphl8Tbtwplv2pmrcGakDMKHSgTa7flRJBtluIEpmNsPjWnkl7Tf.jL4cueL9QAB0n8uCPlEWl2 KGeoWOcSf120Mg_ZiX3q6BqcIU9y7d0KZUO7Bne8maFXsEVCMrEYiw02xBtOp0ll82WDZScmly0y OqBJMl4wIFn7S.D38c7WfRhgCKXBq1ta2F2MMV_pvvl0.b18Iw7iwG_sDzcqo5wzBu150qf5FzuC .sC9hrDnMOeQ4Ybcc_pgIKe69UtleJKJYfq0G_JgHIm_lVvFS7OcNxcaFcWaM5Hls.5Y3zdgbAJD sO.Dm6gf6zsEDfMFy_QEfat13qcTIqgo8yqjCdlQtYAH78swPFW4XeO1WP3tPDacNPi2ZQ2cpnWR 4hRKKpmpVb8ZH7jDMQlholTJGjSduY.p2IMobfNWuIECTJlsUo66dhL3i5YIrAhD0tGn2eqArOe2 BPpWYB57t.7R4.iz7T.uQXFjvuK3TQM19Iq1Z0rmqaK18UB_s9UjVkfjPUpseECOwWkSm6IdCxiW xTwOS3nNFGrk1bLHHa3B9PLeVv55PGnFTYhzKalU7k6HzXBdHN6KG3.KN8H8JJGc18.bff8EeuPN QTUzLczVXM4vfWp_dMZ4dyARBrrKP774GvBjN8u7MUmjWoBl4dhdkKUheKas.fjfp6TD_WX6XWJ. chzj3EBHBLCmSJliSDtuho8KrVAKave4r50vStto1AUc- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Tue, 15 Jun 2021 10:02:10 +0000 Received: by kubenode520.mail-prod1.omega.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 79e69d0cbf0764171f5a0a1c39484f89; Tue, 15 Jun 2021 10:02:06 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.100.0.2.22\)) Subject: Re: Restraining poudriere In-Reply-To: <3CDFB92D-00A4-42E6-891C-B31F10C4842F@yahoo.com> Date: Tue, 15 Jun 2021 03:02:03 -0700 Cc: freebsd-arm , FreeBSD ports Content-Transfer-Encoding: quoted-printable Message-Id: <73E2FF0F-C814-4F0E-AEA2-DAFB0026C4AA@yahoo.com> References: <20210612172957.GA71089@www.zefox.net> <20210612175704.GC71089@www.zefox.net> <16D4307D-FCAC-4027-A41D-F1BD7265D3FC@yahoo.com> <3CDFB92D-00A4-42E6-891C-B31F10C4842F@yahoo.com> To: bob prohaska X-Mailer: Apple Mail (2.3654.100.0.2.22) X-Rspamd-Queue-Id: 4G43l81nXHz3GFT X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=HunGKj8f; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.204 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-1.50 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.68.204:from]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(1.00)[1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.68.204:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.204:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.204:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] Reply-To: marklmi@yahoo.com From: Mark Millard via freebsd-arm X-Original-From: Mark Millard X-ThisMailContainsUnwantedMimeParts: N On 2021-Jun-12, at 18:35, Mark Millard wrote: > On 2021-Jun-12, at 17:53, Mark Millard wrote: >=20 >> On 2021-Jun-12, at 10:57, bob prohaska wrote: >>=20 >>> On Sat, Jun 12, 2021 at 07:36:48PM +0200, Michael Gmelin wrote: >>>>=20 >>>>=20 >>>>> On 12. Jun 2021, at 19:31, bob prohaska = wrote: >>>>>=20 >>>>> ???In playing with poudriere on raspberry pi 3 and 4 it seems to >>>>> work well on the 8 GB Pi4 but is over-optimistic on the 1 GB Pi3. >>>>>=20 >>>>> Can poudriere be configured to tackle packages one at a time? >>>>=20 >>>> Yes, see poudriere.conf: >>>>=20 >>>> # parallel build support. >>>> # >>>> # By default poudriere uses hw.ncpu to determine the number of = builders. >>>> # You can override this default by changing PARALLEL_JOBS here, or >>>> # by specifying the -J flag to bulk/testport. >>>> # >>>> # Example to define PARALLEL_JOBS to one single job >>>> # PARALLEL_JOBS=3D1 >>>>=20 >>>> -m >>>>=20 >>>=20 >>> I perhaps misunderstood what was meant by "builders", confusing it >>> with threads. Or maybe cores.... >>>=20 >>> Trying it now, hoping to see parallel core use.=20 >>=20 >> You do not seem to have mentioned use of: vm.pageout_oom_seq=3D >> (just vm.pfault_oom_attempts=3D"-1"). You also mention "[with] >> OOMA turned off" but no combination of settings actually >> completely disables the possibility. >>=20 >>=20 >> Based on notes in my poudriere.conf for a 2 GiByte RAM >> context: >>=20 >> #NOTE: on 2 GiByte RAM contexts I've used: PARALLEL_JOBS=3D2 but >> # two llvm*'s are likely the biggest combination that >> # could occur in my context. lang/rust or other even >> # larger build contexts need not be appropriate. I >> # normally use ALLOW_MAKE_JOBS=3Dyes . >> PARALLEL_JOBS=3D2 >>=20 >> So for the smaller RAM context: PARALLEL_JOBS=3D1 is a possibility. >>=20 >> On a 1 GiByte RPi2B v1.1 (armv7) I've used the combination: >>=20 >> PARALLEL_JOBS=3D2 >> MAKE_JOBS_NUMBER_LIMIT=3D2 >>=20 >> so that no more than 4 generally active processes in >> builders/JOBS overall. You have used MAKE_JOBS_NUMBER_LIMIT >> before to build www/chromium (2018-Dec-18 report): >>=20 >> QUOTE >> On Fri, Dec 14, 2018 at 05:59:21AM +0100, Jan Beich wrote: >>>=20 >>> MAKE_JOBS_NUMBER_LIMIT is a user variable, so you can either set in >>> make.conf or Makefile.local e.g., >>>=20 >>> $ cat <<\. >>${__MAKE_CONF:-/etc/make.conf} >>> .if ${.CURDIR:M*/www/chromium} >>> MAKE_JOBS_NUMBER_LIMIT=3D2 >>> .endif >>=20 >> Setting MAKE_JOBS_NUMBER_LIMIT=3D2 allowed www/chromium to compile = successfully over >> several days. The -DBATCH option was used, in hopes it'd fetch the = right options.=20 >> END QUOTE >>=20 >> As for allowing 4 processes in a build per builder >> (a.k.a. per JOB) generally (for the 4 core context >> without MAKE_JOBS_NUMBER_LIMIT in use) . . . >>=20 >> # By default MAKE_JOBS is disabled to allow only one process per cpu >> # Use the following to allow it anyway >> ALLOW_MAKE_JOBS=3Dyes >>=20 >> So with PARALLEL_JOBS=3D1 that would have a total of 4 >> processes. >>=20 >> I'll note that threads is yet a separate issue. For example the >> llvm linker might use 1 or 2 more threads than there are cores. >> (These happen in one process.) poudriere does not have a control >> over such tread usage by programs. Threads may or may not use >> up significant RAM in total. >>=20 >>=20 >> I also override a bunch of MAX_EXECUTION_TIME_'s and >> NOHANG_TIME: >>=20 >> # This defines the max time (in seconds) that a command may run for a = build >> # before it is killed for taking too long. Default: 86400 >> #MAX_EXECUTION_TIME=3D86400 >> # Cortex-A53 and such are slow for the purpose, allow 4 times the = defaults: >> MAX_EXECUTION_TIME=3D432000 >>=20 >> # This defines the time (in seconds) before a command is considered = to >> # be in a runaway state for having no output on stdout. Default: 7200 >> #NOHANG_TIME=3D7200 >> # Cortex-A53 and such are slow for the purpose, allow 4 times the = defaults: >> # Also: boost-libs seems to have a long time with no output but it = making >> # progress in parallel builds. >> NOHANG_TIME=3D28800 >>=20 >> # Cortex-A53 and such are slow for the purpose, allow 4 times the = defaults: >> MAX_EXECUTION_TIME_EXTRACT=3D14400 >> MAX_EXECUTION_TIME_INSTALL=3D14400 >> MAX_EXECUTION_TIME_PACKAGE=3D28800 >> MAX_EXECUTION_TIME_DEINSTALL=3D14400 >>=20 >> I use: >>=20 >> USE_TMPFS=3Dno >>=20 >> in order to avoid tmpfs competing for RAM in these >> small RAM contexts. >>=20 >=20 > Something else I will note: you reported for the > 1 GiBYte RAM RPi3B use: >=20 > QUOTE > With OOMA turned off and 6 GB of swap > END QUOTE >=20 > Does the system generate a warning about being mistuned > when the swap space is added: >=20 > warning: total configured swap (??? pages) exceeds maximum recommended = amount (??? pages). >=20 > Such likely would contribute to "swap blk zone exhausted" > notices. I avoid setting up contexts that generate this > warning about the configured swap by avoiding having too > much swap space for the RAM available to manage it. >=20 > As covered in multiple past exchanges, attempting to adjust > kern.maxswzone to deal with this has other tradeoffs (less > space for other kernel information) if I understand right. > It takes more knowledge than I have to know if such tradeoffs > are appropriate for a particular context. >=20 > My memory is that when I move the Rock64 media to an RPi3B > for an experiment, I use a 3 GiByte swap space to avoid the > warning. (I have a 2nd swap partition that can be also put > to use on the 4 GiByte RAM Rock 64 that goes unused on > the RPi3B. It is rare that I do things sort of experiments. >=20 I took a look via https://pkg-status.freebsd.org and the FreeBSD servers have been building aarch64 llvm10-10.0.1_5 since mid-Feb. without such failures. (Previously llvm10-10.0.1_4 was building.)=20 = /wrkdirs/usr/ports/devel/llvm10/work/llvm-10.0.1.src/lib/Target/Hexagon/Di= sassembler/HexagonDisassembler.cpp is used. It makes me wonder if you have a cached distfile that is corrupted or some such. (The servers use poudriere to do the builds and they do not disable targets.) I do not know if the logfile would give a hint about that or not. I do not remember your indicating what /usr/ports commit poudriere is using --or which way you have poudriere configured to get a ports tree to work with (or related configuration information). =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)