From nobody Wed Jul 05 07:04:26 2023 X-Original-To: freebsd-ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QwrJH4JNLz4m8j8 for ; Wed, 5 Jul 2023 07:04:47 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-19.consmr.mail.gq1.yahoo.com (sonic306-19.consmr.mail.gq1.yahoo.com [98.137.68.82]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4QwrJG2VJgz3kNh for ; Wed, 5 Jul 2023 07:04:46 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=hqLt4OZ2; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.82 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1688540684; bh=HxSWNxOrl3dzXuZWOIoNOy6AtIFbdeDE3Fev3uzkIQ4=; h=From:Subject:Date:Cc:To:References:From:Subject:Reply-To; b=hqLt4OZ2TGieD4FRQgCDgqj1A+y/2HGSRqnJ/Yy2ayXwl+ZfS8R5/Xw0UDppZiLdAO5WzOuDA3EMg17rcI8FqdCMs0UbaNGP1ebxYD5t1DIS2I9CLoAx3/bTtbEJuM/1PGoeayT5uF9elI9fnG0r36LPqUmSh6z3QFsoiJR3GMhqUN0F8fXI+4jt149yM3JJ1c5PTKf28aSUZWZ18yx3EkhOlwyv672a6zmcyQczh8Clheq7AAp8+IV+0O6+wIIPj5cnPytjVFVF2PZzQW2XT4AQBwWvizW0Z7XEVEeRgUKD/u3zEL5EoQlWyAup74ciLeOFHei4ctElSRb4GOxznQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1688540684; bh=YFD4e1hPhGvz+v8dgEamrlgHePrhGRrgsXnJkM8gZbf=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=J7MYl0Hbr0p8WhQ/kENahfWxlGtTbQSiQ0fczNVH5+6QJ0i4UNmRgLSTfhe+WmtNquDafZ4DkGBP1A34ro/wi7Vz/qP7jnbLfgC1Ue0BUaHnseeM5glior4NTsn3pUU52+WSWnZh1nBK2uHn3taOtYfI0mUe/iGYqKqcsa55Wh0m6PKnaOuFY3kTJrKP5Bqngh7d9BWOA7jgG1mxZpjupBoNKj/UxelMjaY2TMFJiKPiItQvjOMFKI39cfz1531t4OuGjWcH1LdpmCFG9TaaRjT2ELJT0Kndxw6abc4d7cHIiBPrHGq8R5GvCsE50l24d+mqJXNjPldXG6G6Wuk9Tw== X-YMail-OSG: yBGjTmYVM1n7ub9UR5zY0RuJ8Vn6jGNawASZx2vLrWtnlNpygkapbW_bnSLWI_A 09D16f9_wv3f6iQOGB7RP_ndDCh2CBf22erswiKmc9Ht5hBmDGI9zH261oI3w8JjqQqxh4JbB0Du 1bW.oPyk4gojHyHPj_8uM3emU78VyHG5y.2e.ayz3z_JjV507Ne9s6cMBcXe0qKNho.u0udG7bH1 ZHRaCbPthvKkYkDki4Bq_bbSleMmmKH1kyvbfONm8AYKfIyU311wc2RXlRdUjbkSLElpe8HA_xQl tOmE6dE6g_qk9lINmT.0os5L01cRPbzS62W9vhT1Z4D7UKNi0gySUvARwVFLO1CBhnnKQQjoEXXE rpLRQ3X9sMRSiThH7JXIW.8bLIJclH5sBB8rWSpeggO8VsMxB.RKatVW35HNDpoUniZwaDfZXjQN 5ahHG7OOWj0_x53feITtbHeDfb78u_3gHVElgjdi6PQl0s3MgfbWVQiHUbATLVHFcP6JufINstin GrukpQRO31TN08GCW5Ltj4GOfppI.5OcalgLgi9t7djzXavlL6CmGgELB43U02Q0fWPBrOU0J6Bp NcFRD7N6O8rz.TfiJYtjG0do.zeME9s_rpOPv38N8pOUzcgKMVYlGb9dyfU8A0sufXtUYeHDJPoP 6o_INBTaJe_r1N7Yjuvj_urGzjUyYpRZm6XGROPjXKADyaj_qlPqXyo0p4Gmp07EP0IQy1XH60Hr xuaC6PxX6osJyMVmW7nlDuU06qoQldSuI3wkoW3fpg4iEyJNj8IMnry8m.aLM8PpkipwyNvajFQG BOAMEBJ3D3dMH0rsmmVh0s_YwgqplAPziB7G33h2072W8gfgmbnKzQarupJijpV7y5sAoKw2tap9 R_jUyVjDcY6LPDfdyx.zsQjf2E0BLnRHRzv7coS7Yp0dPC8_mmHrWl_YsD5PbB6PAnK6ID9qm0EU bqfBGaKNmH9VPVh6hS_AVhDSZstn1ibFv1Oeg3YaWJsPIpVoWjnrQ4GV8ruXIjWqx6c_GoiFOwdE zd6uX9Bn4_CXc5QPGETEHdNo0sVyLTYNHRN_ggZ4eK7eAV9C8UlEPj2S2G.Nc45Xdn6142MgwkLN x3kzP6wfa5Qyy41YmBnZIpM.LjiSPnzRbj5EGINd2SYrYSEcsurX40pwdlMFsNiidoQF8.7eDDNz rCE5YP9ERO3y3d24cQ.r0BMdGAaIo5A8tnxHmT0YaynstQzsIYpnIWSYt3iNaGUv.FC3dgl6Azyb CwonyepLy0LUZaeEFbzeUx6ssCh48piHvF.Bcd_1pffzOQAG9P4Y1fX91_EhPkzsFWTwTQ83A3tl aZ3b3HrtNwsTFap2SjS9zkoy84ThV3Ov4tF3OQXp4b_gajT5b07kZUGvvIgaSZSeS0LmP6haneIR El5M1ZwF58uk91ug7yPBjuLtQT27QXRWq9UtV5gg7oG1DVqzMgT2sFpwTA5JckwqoNNnAivne6hv jJrMriPyxRhu10o37cq4crkuZJ62pZUVij92W9lHRW0dGp.FhUviYisjnpFg9DoughVRIyy3PeHz 68FmLHSftF7u8vgpqxp9EjfQVVMnMsgwbxVyUxeUOzZcB8m1vXlxx.1NmCUMjr.UhgecoJhLIi6L VX24nLmW12OyaWvGupmo6Mgdlk5h9_KqCYRgoIjUYJ056Bm2kbjrRPGrAX8x1J7BCG5rAaLxpQua mlFmxmZErEIJH4b4p3c4RKh5Nabiwy4Cv3CH9_HGENZjx0WHblSnOHsVRwD2OJbt93fNV8snEsr3 PyXyvaxRvj4bVDtz0ulKCeo57ZcVfmgoFzeAZx.o8ID66P8T6B90CrXcH13whyMucZIqwardUpPu x0aCF.MYW6p3aoMEcBUAYTvUQ0nseDVXlGlBqrxdFemHZYPpD3yupDbPLfv7jDNxPluvVlM9juGI EyToQWyGZH0lOgK0bMvSg3MywK2IAs0Ji_8Jr0gH8WWN5eb22eOs7nomh5wPmRZFoUzKZJRNStFv QAXkLW5zs38DU5OHYFiMcVwwQqxgijS_iGlxyQLQHZ.vRXWFMauewB5h8d5KDJGRpZ1qFtWaR1y6 NcYPjjMoYWcAMhj9X6ee5rRrY4Cs1CoQF.W6ObSkI2mo8pvnVmFcOjg0sD.60yI2hA6TwOASqoi3 DcJ28jyjX_RdmPmVBEfiVZPXw_1pFJTpUvC3w0.JmCYSShEBuHdLrET.eErlkP0LVDNgJK0ShL2y cLu0nT.TMxoiFfds_Hw7uB2YA_97ETYaO2_jktCxiLcHmdRLY47S.FqvZXnGVlLFoa20rVJXYzma VzKwl2UEczFDh X-Sonic-MF: X-Sonic-ID: cf75972b-70f1-4a08-9f0e-cf6a02d00da3 Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Wed, 5 Jul 2023 07:04:44 +0000 Received: by hermes--production-bf1-5d96b4b9f-ngknc (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 843d89826f3499b45a04d01c4c654cc1; Wed, 05 Jul 2023 07:04:38 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.600.7\)) Subject: Re: More swap trouble with armv7, was Re: -current on armv7 stuck with flashing disk light" Message-Id: <9C960C22-B392-41D9-890C-00FC85D5BB29@yahoo.com> Date: Wed, 5 Jul 2023 00:04:26 -0700 Cc: Tatsuki Makino To: bob prohaska , FreeBSD Mailing List X-Mailer: Apple Mail (2.3731.600.7) References: <9C960C22-B392-41D9-890C-00FC85D5BB29.ref@yahoo.com> X-Spamd-Result: default: False [-0.58 / 15.00]; NEURAL_HAM_SHORT(-0.99)[-0.992]; NEURAL_SPAM_LONG(0.77)[0.770]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_SPAM_MEDIUM(0.14)[0.145]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-ports@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.82:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.82:from]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_CC(0.00)[hotmail.com] X-Rspamd-Queue-Id: 4QwrJG2VJgz3kNh X-Spamd-Bar: / X-ThisMailContainsUnwantedMimeParts: N bob prohaska wrote on Date: Wed, 05 Jul 2023 01:02:31 UTC : > On Wed, Jul 05, 2023 at 08:22:43AM +0900, Tatsuki Makino wrote: > > Hello. > >=20 > >=20 > > It may be possible to set stricter restrictions on selected ports in = /usr/local/etc/poudriere.d/make.conf. True. But this does not vary the number of builders that run in parallel. That must be separately use -J1 (or an equivalent) for Bob's RAM constrained environment. I do not remember Bob P. reporting explicitly about the number of builders that were active. Probably only 1, but I'm not sure. > > For example, > >=20 > > # normally > > .if ${.CURDIR:tA} =3D=3D "/usr/ports/devel/llvm15" > > MAKE_JOBS_NUMBER=3D 1 > > .endif > >=20 > > # pattern matching can be performed > > .if !empty(.CURDIR:tA:M/usr/ports/devel/llvm15) || > > !empty(.CURDIR:tA:M/usr/ports/devel/llvm*) > > MAKE_JOBS_NUMBER=3D 1 > > .endif > >=20 > > # not limited to /usr/ports > > .if !empty(.CURDIR:tA:T:Mllvm*) && !empty(.CURDIR:tA:H:T:Mdevel) > > MAKE_JOBS_NUMBER=3D 1 > > .endif > >=20 > > If we write this on an individual port basis, we can use the = resources to the very limit where they don't overflow :) > >=20 Yep: I've got examples around, such as: .elif ${.CURDIR:M*/editors/lapce*} MAKE_JOBS_NUMBER=3D1 > I just tried to turn off parallel jobs entirely by omitting > ALLOW_MAKE_JOBS > from /usr/local/etc/poudriere.conf What of the number of builders? Was this a "bulk -J1" test? > The machine ran out of=20 > swap as usual, in about the same time, despite having only > two processes running that were visibly related to poudriere > with a total size of ~250 MB. The number of threads roughly > halved, but the time to swap exhaustion didn't.=20 I've not seen anything about the devel/llvm15 build options enabled vs. disabled. Were you using a "bulk -J1"? > While poudriere makes /devel/llvm15 by itself, top reports >=20 > last pid: 15623; load averages: 0.88, 0.88, 0.75 up = 0+00:26:27 17:39:54 > 34 processes: 2 running, 32 sleeping > CPU: 27.9% user, 0.0% nice, 7.3% system, 0.0% interrupt, 64.8% idle > Mem: 274M Active, 219M Inact, 177M Laundry, 221M Wired, 97M Buf, 22M = Free > Swap: 2048M Total, 1032M Used, 1016M Free, 50% Inuse >=20 > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU = COMMAND > 14989 root 1 20 0 41M 25M select 2 0:10 0.01% = pkg-static > 1080 bob 1 20 0 14M 1460K select 0 0:00 0.00% = sshd > 1077 root 1 31 0 14M 1420K select 0 0:00 0.00% = sshd > 1029 root 1 23 0 13M 1600K select 1 0:00 0.00% = sshd > 1042 root 1 20 0 10M 1616K select 0 0:00 0.00% = sendmail > 14988 root 1 68 0 10M 2976K wait 0 0:00 0.00% = pkg-static > 1045 smmsp 1 68 0 10M 1344K pause 0 0:00 0.00% = sendmail > 15405 root 1 126 0 9432K 6636K CPU1 1 0:15 100.04% = makewhatis > 1081 bob 1 48 0 6852K 1016K pause 1 0:00 0.00% = tcsh > 1162 bob 1 52 0 6824K 1016K pause 3 0:00 0.00% = tcsh > 1166 bob 1 20 0 6688K 1784K CPU3 3 0:05 0.30% = top > 726 root 1 20 0 6612K 1348K select 3 0:00 0.00% = devd > 11515 root 1 68 0 6568K 2812K wait 3 0:01 0.00% = make > 1399 root 1 68 0 6212K 1712K nanslp 1 0:33 4.92% = sh > 8353 root 1 53 0 5820K 1632K piperd 0 0:00 0.00% = sh > 1099 root 1 20 0 5820K 1624K select 1 0:10 0.00% = sh > 8360 root 1 68 0 5820K 1588K wait 3 0:00 0.00% = sh > 1086 root 1 20 0 5584K 1048K ttyin 2 0:00 0.00% = sh > 11543 root 1 68 0 5480K 1548K wait 1 0:00 0.00% = sh > 1076 root 1 21 0 5424K 1120K wait 2 0:00 0.00% = login > 1085 bob 1 24 0 5380K 1116K wait 3 0:00 0.00% = su A pkg-statics process pair and a makewhatis does not seem typical of the bulk of devel/llvm15's build time. It is unclear to me what to make of this top output. Having more processes than top displays tends to limit information as well. Ordering by "res" tends to show large active-RAM usage processes at the top. One can have a large SIZE process that has little RAM in active use over a significant time. But, with the 40 min allowed for a page fault, it is far from clear how much space is just for pending but in-process page fault activity. > The SIZE numbers in relation to swap used are puzzling. Overall SIZE gets into how to memory space RAM shared across processes and such. There may well be other ways in which the total of the SIZE figures is not just a direct RAM+SWAP usage figure. Having a chunk of memory address space identified but not all of it allocated to RAM/SWAP may be a possibility that could still count towards SIZE. > Shouldn't > swap be roughly the difference between total of SIZE minus total of = RES? No, for example multi-counting of shared memory/RAM. > That's not the case, unless my eyeball math is way off. Not the case: your eyeball math is working. > The poudriere run just failed, in the same way as before, with 1228 MB > of swap in use. So, some of the 177M of laundry may have been written out to swap space. (One of many possibilities for what might have contributed.) > It's tempting to try running make in /usr/ports/devel/llvm15, just > to see if there's a difference in behavior. Was poudriere running only one builder? =3D=3D=3D Mark Millard marklmi at yahoo.com