From nobody Sun Jun 13 00:53:44 2021 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9D6E67E9048 for ; Sun, 13 Jun 2021 00:53:54 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic312-25.consmr.mail.gq1.yahoo.com (sonic312-25.consmr.mail.gq1.yahoo.com [98.137.69.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4G2bgQ14sCz4bjm for ; Sun, 13 Jun 2021 00:53:53 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1623545631; bh=hQHg7ugPyTHEhXOT3B/6+lshu/QEFA1Co0aDkU8WSdY=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=J7cj1f0J7vWuCeR7mVNR5w1vf7SHblviTehmiCm1qnIGCDrct/NKz4VrWMdLLrp9ZLRnwpUM1tuRWw9zTyI1ycyCyRVelJqCHWDqpPHS/O3uUke/RDF0gOaAL8WNjYPYgNZiBKKm79l8JRLqHmdZO6fzWc2jwCJKccrY9RYypwCwhYx+k+B9Mye5v/ehu2xRYl0TfAF+QVw2JH1RrNxOAhuUlnfHu4sGx1T9BhRmVbNmbzeEcwUHVHsbdvTrQQ8+PrgSCp8aLrfxlnk6PnpSY0mbkLklJnKLXfkGAImN9MxW7/FlogTfEC2P5w02V0cPyUvWnlRbdc/CBsWpFB8YcA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1623545631; bh=dfGktU4MHDGwKTz5Ygk2Kj9ilu0o/NIuezwl3exbdxm=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=hPbWXguUWibwIwbLe2y8smmBhvWpwBB8lCjmpuAcsSkI3XHALTR3WOZhNNRXhdI/oD/ZZJbzNkV34i1bIS/+o60v6X27hiKmHnG66lGKxnQVhZjFBnQvF/NqNghjJb/b7S9Ou2OjkMSjycE7tlaPhPfx8EpPIpDcz4eHDIYJMlg9nMymXv5GUNhmXu7Dw3OJVd7WBPivYma10y+p9CMAAv76gKEAoI3+B9mComFLQ57p6kholqEMN4Si+kpOtBWOD0tAiXt4xpuykg0oqx3wqPfBMck0i6dT6hqyyawfozmTkvXxCcKM1+7E8HrcODKmLsyZV+DNlosDAu26wfk9GQ== X-YMail-OSG: LYO9iPoVM1l41N7tzj6YzJVL8aXtyDudMaV40Ju5axl4hTRR_0XhL2ORHLv_g_g FuQvvHW1o0jlx_FmsRp5j1WmwTE5mYsIL7vIdaW9X_u2dCAZ4kq1UyypvP_CpYWf79OdRgjdF6Zn jSHuQWoeAvM4E1N_Gs46SP4gay_GRwI7Yt0tPbkHjkO4seUSd.UjFyGxodpofNHCMFc_2bneOa6k I2NLPdMoTQLfsK6HLlEa4UKYju_orw6uMIlyRFum1CLjqVsB7nprwFenWnkDmAaQqeKc7ZzvYDXx aFufuq5zd_dbWQqYXOLcK._S2M17lfiTvEIah4lECnMmNu92CG9Tbfzp5Hsyd__8ajHfhOw0MLGw H68nw0fa3xUwu7lYF9yYbYUwPqLlytLW8m.6uQf7sEKyNZkgDeHI0IONV_OHLrBMGp0ELk5yqzG8 fFBxMLDwF3D4AdzjxyeoMATmHTvs5o9MkF.eZNgwTAeRrY30dp_goH4cXqAc4p_ACzX_IONbjXzI bQTtQglqxRcv04V4TJnAs3eqWWq_xlXhndPWb67RFaqyd3FqfHpoVOuS0zbs5NNBVBQ_IvDyUuVG fFf3WsmBFRyIVRLKVwQ97JS63PrProiqo3aRMTE32zn2yPHktnyj.C9EV19AfTBBFazcg0rNxPQq wmDrTcDUUCt8ZjiZ2zbAg4LLmaBsPSG.wAEeBbG6XEo5u42BSf3.oMwG8r50ZhW_kZuPwjnpm.C5 AwkW.6nWwuctkhwl.WvJrXoaVK3BmOQw5KvH7v9v56UjHju7Uf0USGcbuxUZi0hW7thljiHD8SNM sTeK8x3mCzCK.84iHJQOaJr_rvmqxrS7FtzX..4eUNYaY5Txm5DHpwOsLr7KPH_TMOXusHKY7F.z GFSw0mCN3a2.eWclZqauOPHANBHImMqNzI7K393QWN7oeXu4M0yJqweZKc5yJXEeRKqzyacao_eL 1hFSDC4iIcxblZ4TbWbTwp_29uXo5JtxCf_K2H3IVI6cXdIqIhOqFHMg0G4YI7lM_K8YDqVIVsdN SV4kJhFcxJIuA1IeWd31_bAHyd6FLWud4JDwI_vefva9YXwukWF6yLarXRCLB6EF3jnEUqA2d7gf uBifyPhYt4E7aW3fZZ7l0bq1mIo0AJdGH_P3fgm2FWfKCtsfUoTvWD06YJqw3yy9QZhJwjH7HKZJ N4FYkI1d3Uq3DQi6qWncl17dENkEsiOhuRUfVPcD0QsWtuO8PyPJKD5cPZV.UDwKFjba3O86ZZLU HgHrbVLGAT9GKVR3QscactAgxWj_ssxBS0bN39iE0knxevSKHVq87dxr1HhAnQWLkzvv8b3M12Uz MP3jRn6CQRxWzDSWF9OSNVJvj4YTWGVDolzCAjgD6owFhhyYX11wRz8f15SKSfjz4FFO05Qm7l6v 4i9XaCochKY1.RwUNM4VA2VV_qfpg._V80sDsOD18JHMFoJ4EkL6yPHmHdncZGBC4iDJKt5CGfjq t46rgdE082RFiaonCQPthlKltX7zsalAAMlu6VsxwtVHDRDBuhY9Kxj_hIDF36jaRGJDm2Q1sZOy CawUHpoSr4w7j0UydizWgx8O8S.IXCmmPw3MuncblGcD2NUzaJ0vCsgyVzK9fF2q3rWTmJe1m5Wr paWEQ1zZkhoJk.dJVzgLXnvTzF47H65Sbr8XTjq29u.Kl7HdoX6PGwz3Pmt5pMm7ccNvpP4gJAxx iBr_.FpiHygXlb.gkzkx15GHXIXXzEoVAq146Be.I9BAVa33uikXPq4GWn31xa0RukiGaRXmcYv6 YzyRinBP7jb2vDBQ2nZhchoSCevZ4ygmzUBU4sMYyAqypbpxROJhj3bfXwA4hzNvQaaHT3ZF.Zxa P58z1g3ulbkZTLS4rvQl05M5uvZHGfUDNvQu5E6DLG8Jcz5vdJG_jwxjosHD67N9hc9pxwRtXNPb Ldh.jLAN_LSdNRfjZhKiktOsqP38oB1q0Zqh86IW0IXbyztXgPrKv6CYmyaKsuUpd2lJj6Hs0S5l HEMQlWKdqSvZk32eh1AFRZh_2arHFgUakNMe9VqUsMvFZE4ML3d5M8ReUpa_ajD2grLKJhh_Pktu Y59DjBlj.H6hm.55uUCUhHJuczkc2RXV8IyYEPpz1RL6mpS6P3L_M9LxWpxYXchv8rDDmO1NoCkK R.wK2Yf1tSVCt_DRA9dTT922XFH52tMYc5mu0Y70TXvRV3VNsYzD7dq7abWVKT7HfwKZ.I88.JVQ V5n5LLPbOoJNwIInbSNdGy_EMsqV6dIFsrRyGTFzYBpMBDeqDFgvXZj2dCZoVJKWMX51OqwQL4HO 6q.zwA3ngRoEGWKijUFs6El_N7FconAC5u0n86hdumKFLilabNfL59fd55NThVTnxbBNMy1UluEh mPBEPCxL2ML2GWlJHb0WjQEzN0s13Rsv6KBHGXGZ9aUcFnVMrfcB3qSvHJdQBIHKIe6zk9dEkDik 95eAwTQDfxN8rQPcNuXCInqJsGESu0ePmo3mldQEVx_DdM_z2jIys4.nlnsXE0SRQGtHoChbA8i0 qVEPsJgEyE0kzEW.UyFMdwsi_bVMP.qerwbUOJtTdS_G9y1fhJCCC6WdM1rANIWG9N.ow50w8iAe hJ8bpK036a0GO6zOkifku78xkEY05i4kjDKm6v1wYUBSBizhLYA-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Sun, 13 Jun 2021 00:53:51 +0000 Received: by kubenode511.mail-prod1.omega.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID c81012ee855044ceb7b6969436801447; Sun, 13 Jun 2021 00:53:46 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.100.0.2.22\)) Subject: Re: Restraining poudriere In-Reply-To: <20210612175704.GC71089@www.zefox.net> Date: Sat, 12 Jun 2021 17:53:44 -0700 Cc: Michael Gmelin , freebsd-arm@freebsd.org, freebsd-ports@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <16D4307D-FCAC-4027-A41D-F1BD7265D3FC@yahoo.com> References: <20210612172957.GA71089@www.zefox.net> <20210612175704.GC71089@www.zefox.net> To: bob prohaska X-Mailer: Apple Mail (2.3654.100.0.2.22) X-Rspamd-Queue-Id: 4G2bgQ14sCz4bjm X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Reply-To: marklmi@yahoo.com From: Mark Millard via freebsd-arm X-Original-From: Mark Millard X-ThisMailContainsUnwantedMimeParts: N On 2021-Jun-12, at 10:57, bob prohaska wrote: > On Sat, Jun 12, 2021 at 07:36:48PM +0200, Michael Gmelin wrote: >>=20 >>=20 >>> On 12. Jun 2021, at 19:31, bob prohaska wrote: >>>=20 >>> ???In playing with poudriere on raspberry pi 3 and 4 it seems to >>> work well on the 8 GB Pi4 but is over-optimistic on the 1 GB Pi3. >>>=20 >>> Can poudriere be configured to tackle packages one at a time? >>=20 >> Yes, see poudriere.conf: >>=20 >> # parallel build support. >> # >> # By default poudriere uses hw.ncpu to determine the number of = builders. >> # You can override this default by changing PARALLEL_JOBS here, or >> # by specifying the -J flag to bulk/testport. >> # >> # Example to define PARALLEL_JOBS to one single job >> # PARALLEL_JOBS=3D1 >>=20 >> -m >>=20 >=20 > I perhaps misunderstood what was meant by "builders", confusing it > with threads. Or maybe cores.... >=20 > Trying it now, hoping to see parallel core use.=20 You do not seem to have mentioned use of: vm.pageout_oom_seq=3D (just vm.pfault_oom_attempts=3D"-1"). You also mention "[with] OOMA turned off" but no combination of settings actually completely disables the possibility. Based on notes in my poudriere.conf for a 2 GiByte RAM context: #NOTE: on 2 GiByte RAM contexts I've used: PARALLEL_JOBS=3D2 but # two llvm*'s are likely the biggest combination that # could occur in my context. lang/rust or other even # larger build contexts need not be appropriate. I # normally use ALLOW_MAKE_JOBS=3Dyes . PARALLEL_JOBS=3D2 So for the smaller RAM context: PARALLEL_JOBS=3D1 is a possibility. On a 1 GiByte RPi2B v1.1 (armv7) I've used the combination: PARALLEL_JOBS=3D2 MAKE_JOBS_NUMBER_LIMIT=3D2 so that no more than 4 generally active processes in builders/JOBS overall. You have used MAKE_JOBS_NUMBER_LIMIT before to build www/chromium (2018-Dec-18 report): QUOTE On Fri, Dec 14, 2018 at 05:59:21AM +0100, Jan Beich wrote: >=20 > MAKE_JOBS_NUMBER_LIMIT is a user variable, so you can either set in > make.conf or Makefile.local e.g., >=20 > $ cat <<\. >>${__MAKE_CONF:-/etc/make.conf} > .if ${.CURDIR:M*/www/chromium} > MAKE_JOBS_NUMBER_LIMIT=3D2 > .endif Setting MAKE_JOBS_NUMBER_LIMIT=3D2 allowed www/chromium to compile = successfully over several days. The -DBATCH option was used, in hopes it'd fetch the right = options.=20 END QUOTE As for allowing 4 processes in a build per builder (a.k.a. per JOB) generally (for the 4 core context without MAKE_JOBS_NUMBER_LIMIT in use) . . . # By default MAKE_JOBS is disabled to allow only one process per cpu # Use the following to allow it anyway ALLOW_MAKE_JOBS=3Dyes So with PARALLEL_JOBS=3D1 that would have a total of 4 processes. I'll note that threads is yet a separate issue. For example the llvm linker might use 1 or 2 more threads than there are cores. (These happen in one process.) poudriere does not have a control over such tread usage by programs. Threads may or may not use up significant RAM in total. I also override a bunch of MAX_EXECUTION_TIME_'s and NOHANG_TIME: # This defines the max time (in seconds) that a command may run for a = build # before it is killed for taking too long. Default: 86400 #MAX_EXECUTION_TIME=3D86400 # Cortex-A53 and such are slow for the purpose, allow 4 times the = defaults: MAX_EXECUTION_TIME=3D432000 # This defines the time (in seconds) before a command is considered to # be in a runaway state for having no output on stdout. Default: 7200 #NOHANG_TIME=3D7200 # Cortex-A53 and such are slow for the purpose, allow 4 times the = defaults: # Also: boost-libs seems to have a long time with no output but it = making # progress in parallel builds. NOHANG_TIME=3D28800 # Cortex-A53 and such are slow for the purpose, allow 4 times the = defaults: MAX_EXECUTION_TIME_EXTRACT=3D14400 MAX_EXECUTION_TIME_INSTALL=3D14400 MAX_EXECUTION_TIME_PACKAGE=3D28800 MAX_EXECUTION_TIME_DEINSTALL=3D14400 I use: USE_TMPFS=3Dno in order to avoid tmpfs competing for RAM in these small RAM contexts. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)