From nobody Sun Jun 13 01:35:01 2021 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id B51607ED1D7 for ; Sun, 13 Jun 2021 01:35:10 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-22.consmr.mail.gq1.yahoo.com (sonic310-22.consmr.mail.gq1.yahoo.com [98.137.69.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4G2cb13msyz4h7w for ; Sun, 13 Jun 2021 01:35:09 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1623548107; bh=Trpgz2ujKT+FVwLfu7q1PoydA5TX0Ai83srzqWw6jbU=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=gmXCVU8AGa2dH0gjKlgEDfhbz/XvY2G5YDb26QrPBHtV55p3A5NKGYSCbbd6gWZVr87nsoxEs5zuxjq8znfMw2YTuFTCkiO2RFdf6/2f/fM4dKXj/5IQrs6vjs6e+Kb2OauWjjm6LnnvZuGVY4CfTFUwqIadYQf/PuVrrrezHC4WfK1PmTGnW+RFia7lvg1GONKPxXXudkqUH/ByoTxblty9rZgDIC54kdrUwTrG4LJFVpWbySDWPaBGhreOFyun25swAFd6Cy46R2qTit1WIRmNej6NjDytR0aWuXv9ylIieS8/RvnfOui5Z2dr4JMISC1EFkO5CteETNpwk7pnHw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1623548107; bh=I054auOGKeWsOFRAGiYNbsFiXjynU1fh1BdQSxbf1L1=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=a2Da9jxVQ3gIRzslTP/QkIS0G+MDlGZP9twOul/DI4OEvJ1OgnL1lLeBM+B/MkKdTyoL2fMDNofePF74CQ31jgFbVd0G+wrah2UJY1FMHW5f9+kKK7KA7dAp7o8oxSxVNc3hp/39Fn+UpHfbbQPriGi9Tkl5nD+b4+eM8+C9tGtkK5yGhsFhClNUEpjejLKXb+/driFFsUWut9vNE6HbVyWfNAWu7z7X2+x95XMHGqGZdLxdGkH1bqUqPp4nIb7y8vVTaMl5wAymEG2ABSxsdexeA52eAnFXfURVxjkKrRDfjXMiskfsajfZT6Aca230QCv9agYFfKO2T4Cb46AK/Q== X-YMail-OSG: sAvUqFUVM1k0n7n5G4hbH6O7IwK.ob7Ca38BZb.BRC_j3vUKuD5w_WTg7kCzk_c 02zFEEszuQ5H73Q.uFy8o03s63HhycYHYo029ORbvUR5_ahkkJ3TekqJZtfLs9mUa7I2um.WIsu9 nsbSMrxxyk8gl7WtaciWvwncsz1547yr4kItE159yyvFtuizpnSdqAl8z9A_JbAQ0lak39ToNlOp 1GayNmXNwTMqOZT3zV5YK0pAVETI30pUpkIVRAu0vLAefufsUGkrTg4CttU7UHnP3S0uXW7AmXie MW5K950.PJ2RZrOjUn_fFRuqxW2rTI4f9.6CX0Ujd8CK8oY.75_dzQ.l1MThXqxD4W9vEV5B36o9 3co.nslRKtsurMhvsZSHXivPFuvEigHVOJbrq4gkEIqcfmRnppvKgLaNBCHNaerkWKaZRPUH5Q_6 _IuQSfCdkazWUklyJ_b9ryXsLVSyfQ8JvkyUi4VIKheATgpMo.poD5wJg2_RbCFuYMcFd9JqTc9i 5zKz2Kv3bffy4Xv9Nzrf.pMpO7kotWefjiPnN.bH5677J_fPo64J3EChVoCfkXuZ7V88a_xjSByE ir_bgCSMLWxRbV4vqvwyau.1K7n_qay.Q_EKjPUDzMMCXwyC0fLloH.c57tdj4zF07nBYPrhyOC1 C_ccjfG44KhPlC133bsOD05R9dbsB2erzBJGAGw0O97iDKfv352pVOR8xJ07nEn4VOQxWwMzmTdh hupyMOnXhNbkM._nX14FSTx9IILjxq5ZSDbnYKeGG9O_QaioSSGa0i_e4vbW7XVxOcZRmTCgkR5U NZNWCIdkFy12WITR2o7moi2Ym_h34x_ccA4PGMci5WZ9A6LdEVr.tVToQDqSG7CeNaojqal6F1JG EInFUsvZ4hvEGAfUb2oPZAAutUvOtAyPJONOmByq2oxFi2bMDD8S_9yoUrNZger5nmVcZf0qiR5B Qws_EWJtPsH1je_PgqgGzmSCDDBRm6TUC7PuelP2izqyurLpKiBvhmBhV8VqSFf2F.sCwBj0UQ99 1TD0BxpsxbyvNTNJcRvB5h9K2nFvelmCTiLqMu4xpq6s_fQScaB8cvqRkEW7iASm2ytgqe_eXWew 3qxlWukFpQ6IksP8_5aZVJQfc.yITJKV42r5eX28Cxj2wBgEE9ic_mfCOjoUTS39tbyEiSC_0Qwv EaygumWv.rDdlugpKcljP1etDaWKge1km6f5Yx3j2bB7dV3.eOkKy9yXhrLEIoeJ4H_o1qZVW7_E AdVIxrbTdEcZgCEImoG13P6C.LtFcbcvt97yufXYbDpDQPygU1r8tzGolwtw9aJg7cNgINdrmeqF IppmMqctyQ_1iyjETXrs.shQhKcMsKXJlFfMeT9.bd4l4po2gPjuKIIMFS3bqmc3K1w0bHajaRO5 iz.UdGOA_YsxcW5MNcSduz7uO6PPFduQaViodw2u.xeHC39WTLQYfOhIlImFP.FOjlNYznK3O6qJ Iu6n47GsTZJSi3qphatS0pl4jMbgkLvnloDFT7h.E.TAka.v5yTgsX180172QpQVrgFYO1eF0U6E B_o.GJFT12zmF.ez.LBq3T2eDboFfdulVYp0nqXpf9Q8N9w.Q7GaDLfjwwsa8kuyxrEJkSC.m.4f 6qGEe_RX_lar1wbvWevHg_FywOY_OLjBs9LZndoBQw.b5gbi1SK26uMZr3mZm3mPeowfeAbtuFsS I17wxbHrjGyW9GfbsESWFmXp9AAmL3JBDKznGhWBafbHptuy1rDA68vOALMojTdSEBLgDhGknnYo Nayp6EHjEVYtR_gEhq.C99AlMGrY0jERmaf14tXtdxPb7lLlOLmuHtYsPPS5SyBnjhcLaLo1AEq0 EhljdhEG6p6p24h2odRPH47wV32rnF3AjIiA1QGUIlt5bvL2MoaQflntwk2Zjdm6wKBgwdf0Cd3x HgRIAmer4q9ZSXBHSxrpFzeFFuXmPXRGVkPQtvm.Gycx3y2sP9j8k_nlyPD.vJZU0uUo8cyBNOWN YU4ROU87pjBBpbyk_SjhaGFUO5y8o_4Jke8mwWqRBd7Z956DJof7.2L4hFdYH0g4t7oj7Cz7eEuv VBzaZhNmPo5L0wDAsThHLG_14X35BFy40XjkF7IwgxTHWWw0PioI.rztXF3gcMOxfw2MXkg2JJGH cU1FlMfM_AzeMFj5DgLBaFwoo7q55UD0cyTRYY7a9_KCkQT7eduMrGOOwix5dG_rPoikM40YNXvg diYaf8nYGSn84dnI2ic.68nVqK0BFM0m91ShfiQHLov44GcC5mnA_X_g1LpMk5QdhdN.KxUcnBPz kPwdag.8Dvy7U6KpzbBFcKV9leeAVKShbL56Jmf1aRZAt3w74MbNtz8r6BV6SEaYY8xGt674ysCB oItYfJsj1cyUUG8kUCEwJ6t.kY5ZDy.uDrtwxr1SK7Ic3L1xQZlZI_sxDO6Qu6wAOKS7H01v4t00 C4.QzqMmS9ePxYAJrgG.LKPuPWis40W0q6Hbqu_wk86Q0EP1pz0HVv3oO6mBxeYMOC3NAmxAX1tX KDfbTFwYDOi3Gf1.PARDrxB_.uIUKzfR01lPG8YbwpRrqSklDDwiqaPcNjPVPQ5.oxxjQkO56nVg rIodbonrqaUCYFbILJCGUMJjDgh7O4281_qtGJy2EQjPFF8rPDw-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Sun, 13 Jun 2021 01:35:07 +0000 Received: by kubenode520.mail-prod1.omega.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 76a7bf7ff912d97c8fa34549c9879f53; Sun, 13 Jun 2021 01:35:04 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.100.0.2.22\)) Subject: Re: Restraining poudriere In-Reply-To: <16D4307D-FCAC-4027-A41D-F1BD7265D3FC@yahoo.com> Date: Sat, 12 Jun 2021 18:35:01 -0700 Cc: Michael Gmelin , freebsd-arm@freebsd.org, freebsd-ports@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <3CDFB92D-00A4-42E6-891C-B31F10C4842F@yahoo.com> References: <20210612172957.GA71089@www.zefox.net> <20210612175704.GC71089@www.zefox.net> <16D4307D-FCAC-4027-A41D-F1BD7265D3FC@yahoo.com> To: bob prohaska X-Mailer: Apple Mail (2.3654.100.0.2.22) X-Rspamd-Queue-Id: 4G2cb13msyz4h7w X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=gmXCVU8A; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.148 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-3.50 / 15.00]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.69.148:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.69.148:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.148:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.148:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-arm] Reply-To: marklmi@yahoo.com From: Mark Millard via freebsd-arm X-Original-From: Mark Millard X-ThisMailContainsUnwantedMimeParts: N On 2021-Jun-12, at 17:53, Mark Millard wrote: > On 2021-Jun-12, at 10:57, bob prohaska wrote: >=20 >> On Sat, Jun 12, 2021 at 07:36:48PM +0200, Michael Gmelin wrote: >>>=20 >>>=20 >>>> On 12. Jun 2021, at 19:31, bob prohaska wrote: >>>>=20 >>>> ???In playing with poudriere on raspberry pi 3 and 4 it seems to >>>> work well on the 8 GB Pi4 but is over-optimistic on the 1 GB Pi3. >>>>=20 >>>> Can poudriere be configured to tackle packages one at a time? >>>=20 >>> Yes, see poudriere.conf: >>>=20 >>> # parallel build support. >>> # >>> # By default poudriere uses hw.ncpu to determine the number of = builders. >>> # You can override this default by changing PARALLEL_JOBS here, or >>> # by specifying the -J flag to bulk/testport. >>> # >>> # Example to define PARALLEL_JOBS to one single job >>> # PARALLEL_JOBS=3D1 >>>=20 >>> -m >>>=20 >>=20 >> I perhaps misunderstood what was meant by "builders", confusing it >> with threads. Or maybe cores.... >>=20 >> Trying it now, hoping to see parallel core use.=20 >=20 > You do not seem to have mentioned use of: vm.pageout_oom_seq=3D > (just vm.pfault_oom_attempts=3D"-1"). You also mention "[with] > OOMA turned off" but no combination of settings actually > completely disables the possibility. >=20 >=20 > Based on notes in my poudriere.conf for a 2 GiByte RAM > context: >=20 > #NOTE: on 2 GiByte RAM contexts I've used: PARALLEL_JOBS=3D2 but > # two llvm*'s are likely the biggest combination that > # could occur in my context. lang/rust or other even > # larger build contexts need not be appropriate. I > # normally use ALLOW_MAKE_JOBS=3Dyes . > PARALLEL_JOBS=3D2 >=20 > So for the smaller RAM context: PARALLEL_JOBS=3D1 is a possibility. >=20 > On a 1 GiByte RPi2B v1.1 (armv7) I've used the combination: >=20 > PARALLEL_JOBS=3D2 > MAKE_JOBS_NUMBER_LIMIT=3D2 >=20 > so that no more than 4 generally active processes in > builders/JOBS overall. You have used MAKE_JOBS_NUMBER_LIMIT > before to build www/chromium (2018-Dec-18 report): >=20 > QUOTE > On Fri, Dec 14, 2018 at 05:59:21AM +0100, Jan Beich wrote: >>=20 >> MAKE_JOBS_NUMBER_LIMIT is a user variable, so you can either set in >> make.conf or Makefile.local e.g., >>=20 >> $ cat <<\. >>${__MAKE_CONF:-/etc/make.conf} >> .if ${.CURDIR:M*/www/chromium} >> MAKE_JOBS_NUMBER_LIMIT=3D2 >> .endif >=20 > Setting MAKE_JOBS_NUMBER_LIMIT=3D2 allowed www/chromium to compile = successfully over > several days. The -DBATCH option was used, in hopes it'd fetch the = right options.=20 > END QUOTE >=20 > As for allowing 4 processes in a build per builder > (a.k.a. per JOB) generally (for the 4 core context > without MAKE_JOBS_NUMBER_LIMIT in use) . . . >=20 > # By default MAKE_JOBS is disabled to allow only one process per cpu > # Use the following to allow it anyway > ALLOW_MAKE_JOBS=3Dyes >=20 > So with PARALLEL_JOBS=3D1 that would have a total of 4 > processes. >=20 > I'll note that threads is yet a separate issue. For example the > llvm linker might use 1 or 2 more threads than there are cores. > (These happen in one process.) poudriere does not have a control > over such tread usage by programs. Threads may or may not use > up significant RAM in total. >=20 >=20 > I also override a bunch of MAX_EXECUTION_TIME_'s and > NOHANG_TIME: >=20 > # This defines the max time (in seconds) that a command may run for a = build > # before it is killed for taking too long. Default: 86400 > #MAX_EXECUTION_TIME=3D86400 > # Cortex-A53 and such are slow for the purpose, allow 4 times the = defaults: > MAX_EXECUTION_TIME=3D432000 >=20 > # This defines the time (in seconds) before a command is considered to > # be in a runaway state for having no output on stdout. Default: 7200 > #NOHANG_TIME=3D7200 > # Cortex-A53 and such are slow for the purpose, allow 4 times the = defaults: > # Also: boost-libs seems to have a long time with no output but it = making > # progress in parallel builds. > NOHANG_TIME=3D28800 >=20 > # Cortex-A53 and such are slow for the purpose, allow 4 times the = defaults: > MAX_EXECUTION_TIME_EXTRACT=3D14400 > MAX_EXECUTION_TIME_INSTALL=3D14400 > MAX_EXECUTION_TIME_PACKAGE=3D28800 > MAX_EXECUTION_TIME_DEINSTALL=3D14400 >=20 > I use: >=20 > USE_TMPFS=3Dno >=20 > in order to avoid tmpfs competing for RAM in these > small RAM contexts. >=20 Something else I will note: you reported for the 1 GiBYte RAM RPi3B use: QUOTE With OOMA turned off and 6 GB of swap END QUOTE Does the system generate a warning about being mistuned when the swap space is added: warning: total configured swap (??? pages) exceeds maximum recommended = amount (??? pages). Such likely would contribute to "swap blk zone exhausted" notices. I avoid setting up contexts that generate this warning about the configured swap by avoiding having too much swap space for the RAM available to manage it. As covered in multiple past exchanges, attempting to adjust kern.maxswzone to deal with this has other tradeoffs (less space for other kernel information) if I understand right. It takes more knowledge than I have to know if such tradeoffs are appropriate for a particular context. My memory is that when I move the Rock64 media to an RPi3B for an experiment, I use a 3 GiByte swap space to avoid the warning. (I have a 2nd swap partition that can be also put to use on the 4 GiByte RAM Rock 64 that goes unused on the RPi3B. It is rare that I do things sort of experiments. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)