From nobody Mon Oct 31 04:33:18 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4N10df2H2Vz4gbSN for ; Mon, 31 Oct 2022 04:33:26 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic315-8.consmr.mail.gq1.yahoo.com (sonic315-8.consmr.mail.gq1.yahoo.com [98.137.65.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4N10dd2Mx7z3jWs for ; Mon, 31 Oct 2022 04:33:25 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1667190803; bh=es08fjwyjcOznJh96oeHxTDTxvHoxxkDKW+ENypanGk=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=aoNeZe1VSOFnFf+MgvjIxVknbT25ePac2ADU+e4GDouQSAbDAqcOTMho2A83TocLST8R1NhzTzfvqMN3QWiReXPN0hcX96aI6NIxkuvyBSftklFlxujGSfmoxyRYAHVoitlCDjRwkZQIOa+e3go83xMDgBGrkxBHegByuR0IFPosgeDovp2iZ84OD7rmJlf5UtgRuDWsqG2vAIc2KOYw3yFhwByguT3o0L/IuoZqN6Zp1L395pORS0X/4qRpq3bJysFJZRU+xQBM06tSNWwTN6t2wWrXrXfO5h0nUHHxl9B72nNFlME96S7UzZ8npjOXcpLnIw/mbBtmAE9vOX4m/Q== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1667190803; bh=FtYwIKq157euCRB9lzhGHg75IlY4oRid3v8Cn0K5eVj=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=W0M1sWXNA8xMIZ2t52eJJowMQso5BkUmMFfSw/qTXJh4LIn8+t0ilW+Z0RSVj4RH2LQ/XABZNifycKS1e2gPBB57spTSRqTq6jhvfpcCpnXMZc/pVbZgBHjezyNs70mx/opmUSinLKuDNykG8OikSPDxkJYuVsCK4hXNRbVqn1fRnXGP23c9JMvXMQn7jwCBfLw8XKfhKpOJVj7MaN21rd8W5yLaEA5qJU5HQTNGhjlbG4kQLCYtow40O35C63Lj6HDLytNpUkiP0cVPmq78NpEDo5arkbRG1f+BS5xP7KsWjXCckbvUbA+ZU8Mt7JaXrqifHISNuzIqk+W/Sc+DJQ== X-YMail-OSG: yHiqKPAVM1nZGt2EVzoM.3erjAItMZ1zTT6Gc1XMF.raRMz2h5cZyfruE9sFisj 3pKYT_EFZsS4zbdNYRPuZq3XXHnc5YZLhGVOqbHubLXv17pXIvSIG9vlU9b9CJnCU8vyXzw4Hv9g HTiG7WyUnw7gDEkYZsweR0Bi9w6vHsugZ1BWcThvWFkxywQ7maa8M9T6OoKlBFaKoqX54N9HHIM3 D..ho1svg9fAL3SLNnHe2G7u1Dhfl0.p8rMhGHzfBn6ZBj86dwiI_0wPQJYGjc_OUpV2iA0w7e4a mnBvAdd4A7RfLupnaTl4Rqz6K1.8lt636gSoDNvBdOsMz7GQ3gy_u84evWlaZgyscz5mxysM0l9L ZN3Ykd3iQLSIOtXsgu63HhH_02fwcqNENw84pNlIaIDP2CYtLh9O38hCcE9N4FhHmFKWRrUaeHeM Djp.Rp.0qKqaCM7l.mzFFlJi_P0gjD2sTVvOATEU.x_xzBHLsJxX3cMnQtn9NqQ8nCG6veUKHEC3 Tzoju6rmDEiVGkVc9ast6o0ST.0vw4JIG0bkQbmab3lPgCe9HWe1me2ZqPcGHoT9vnVNG_mPzj1V F3ol3dX.__ZppwZyFzA98km5PY0iWytqlZifpSF657.r0fx5MbDg7g04s8aYNqEyNVG5LYeOZRXC mowcwSRGCIklkLrTh3Yf_yfATBxzVvT5ZByt_yisC6FjeghkNfOVerr6dHHNPVYnugFPpESLkh.Z hDxvf43JCdHE4xcdszharUWxwtwx4haLLIPua0VVl5X5ONhDG.6KTN1t_WMxBWIT.7AotXKjxr6k ZTAXJUtdpxn0odFzdAxQz.n3o2qW2cNDrdnFkZjZ41SdDSZuTtypz7EfgU4HgCngh_D6avYHjLd7 LE_Vk4r59eO7ZjaSunlaQS86lQm7rlj039U9GrQjEs9ofuXKC0lITz5.7RgdXQPp3BN4mBwTH7la DKMY7C.R0rAZkFxoTvTqYjgCjzCCkHaNl5UdNEMAU2KdW_TKJeH3ggltceCAwJOk8kKzLTTQEgSO llzmujKncDtH3Y9a2oIwHfZIh4PWr2D4Oqag4aNCu_Q0vofqY_wN_CkkwuPN6UxstCY31ysIH9gJ GXMI.Gkwro31_YD0VMSL9wYHtSQpt86OrfmiuzlZF7Wgww2StTCPLULOdGQWP1Ki5kqxBCByutKB XQAWRtAaiVTx5mL9MSdJgXjjCPW2nsj4JwRvL.XryU5.b37b4wWXdX_MPmX.jFe0KUOznJG0WWf1 0YXJEbBCGIcHj4pxjYUA_OXnymaFsdfJLNfLqdVUK5R.B4w9fUAiJBTyESmSkTpLAJkIBeZPtbse gToLu.xSq5S2_.ccHGQ73QUIOAMcaUCjtrz9TUtMfNM4sLXe1NXOcKP7sIQjABBIKG8ssYzDyj9. 5oow.8DP.9RNA_07q_PXbNCrxILdT8ZR9k7Y1GrFsN8e4FYGogBSXJPMqeyAvIncrKw0wYQZvWZa QPzIO37K3tbTln4XvCd_DSSknRWShbP_lf6nKPOj7AskvDBXi7ol6QZP2msDp6ikLnXLk5b7M9fG 9pMy1Q5WfuJZ0OkEgg.ixMlxxvWk4GYGqTb3lYqRiFwNwnjYzBYGs0vyt8.S_kIgRugKJJ7aYgZs lHdefvlXSLpiTcgwFcLbQY2qBb6Q2Y7.TTS.wLl1ycxifMJfVXYhlbW1oSPAjisKknJhi6HYA_a5 Kph8z7diK3HhCYdi7_C5egrBqHsc8PFx1JQZY2nUnW5ReR6_UeHRNDTZL290U6FRdqPR6ecEP5ag zWjH58mjqwYPRD6IV0jYs7RTwJNa6cbUDmhQQ0eHZXY5nz59AC6JlwvkxIYHnbYBQgoLlGw.ONYb jP2Jr51CvB9B84crywaOX2.zggRUGm6ufb5wp04rokdDXJwtKHTGzdeDasBwvaWvnzvZWdkI_H_W tfB3FLq1.0Obb6URTpbKuR4VMM31iQP10MKGDEL6V6ITpuiHKT0nAaGipUkXNJ9oqKlVKI6bpqn7 UXKq7nkM7uMRMU_wYGSGLDHR2yvMcGbEy4UGwnVfdWCB5apgGllpjExnTn2OBCqhO1o5eyHkbSt8 Ben6qDSEbWX.Q_SI83IYumvtp630r79yXq0IPyeig3bHhOK3jwaUrdavfCouLSUkjfejJZKnznv. hR2er2SqqP3u.0oqIWDZjQ3XdSjRGC2jJmBCx8Sws.aS2oEq2DYjXg5WH0imOYlMVXAiQUUwF5vB T18YLPdfsu1JLaZzZ.LCWkcLyoIo_JcpD9fvtz6x4NzbHfyVAzeLXd3NKjuNO5q.1h.ZWbLbsqP6 E X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic315.consmr.mail.gq1.yahoo.com with HTTP; Mon, 31 Oct 2022 04:33:23 +0000 Received: by hermes--production-gq1-754cb59848-cmwdp (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID a83c721c04cff2f17b92852026dd07f6; Mon, 31 Oct 2022 04:33:19 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: Re: 14.0-CURRENT failed to reclaim memory error in RPi 3B build From: Mark Millard In-Reply-To: Date: Sun, 30 Oct 2022 21:33:18 -0700 Cc: freebsd-current Content-Transfer-Encoding: quoted-printable Message-Id: References: <84376CC9-B991-4BF3-AF5F-0AA09CB28339.ref@yahoo.com> <84376CC9-B991-4BF3-AF5F-0AA09CB28339@yahoo.com> To: Archimedes Gaviola X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Rspamd-Queue-Id: 4N10dd2Mx7z3jWs X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=aoNeZe1V; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.32 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-3.49 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; NEURAL_HAM_SHORT(-0.99)[-0.993]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TAGGED_RCPT(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.32:from]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_TO(0.00)[gmail.com]; RCPT_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org] X-ThisMailContainsUnwantedMimeParts: N On 2022-Oct-30, at 21:00, Mark Millard wrote: > On 2022-Oct-30, at 19:47, Archimedes Gaviola = wrote: >=20 >> On Mon, Oct 31, 2022 at 1:29 AM Mark Millard = wrote: >> Archimedes Gaviola wrote on >> Date: Sun, 30 Oct 2022 13:41:52 UTC : >>=20 >>> I am building a kernel and world in 14.0-CURRENT >>> = https://download.freebsd.org/ftp/snapshots/arm64/aarch64/ISO-IMAGES/14.0/F= reeBSD-14.0-CURRENT-arm64-aarch64-RPI-20221027-769b884e2e2-258837.img.xz >>> with Raspberry Pi 3B (ARM kernel config file and in default system >>> configurations) and compilation breaks due to "failed to reclaim = memory" >>> error as found in the dmesg. >>>=20 >>> pid 91224 (llvm-tblgen), jid 0, uid 0, was killed: failed to reclaim = memory >>> pid 91131 (make), jid 0, uid 0, was killed: failed to reclaim memory >>>=20 >>> Here's the set of the build commands I invoked. >>>=20 >>> root@generic# cd /usr/src ; make KERNCONF=3DARM TARGET_ARCH=3Daarch64 >>> buildkernel buildworld installkernel installworld distribution >>> DESTDIR=3D/home/freebsd/rpi3b >>>=20 >>> . . . >>>=20 >>> Any thoughts? As I don't have any idea about VM pageout. >>=20 >> Hi Mark, >>=20 >>=20 >> Multiple configuration things from what I use: >>=20 >> I use a swap partition (not a swap file!) to give the system >> someplace to put copies of inactive memory pages (paging): >>=20 >> # swapinfo >> Device 1K-blocks Used Avail Capacity >> /dev/gpt/Rock64swp2 3670016 0 3670016 0% >>=20 >> Oh I see, there's no swap partition in the default installation. >>=20 >> root@generic:~ # swapinfo >> Device 1K-blocks Used Avail Capacity >>=20 >> root@generic:~ # top -S >> last pid: 92429; load averages: 0.00, 0.00, 0.00 = up 1+12:49:11 23:41:10 >> 52 processes: 2 running, 48 sleeping, 2 waiting >> CPU: 0.0% user, 0.0% nice, 0.6% system, 0.0% interrupt, 99.4% = idle >> Mem: 2000K Active, 626M Inact, 223M Wired, 97M Buf, 20M Free >>=20 >> Let me try to create a swap partition. Let me mount a spare USB flash = drive for swap as during the installation all my microSD card storage = was allocated in the root filesystem with growfs. =20 >>=20 >>=20 >> where gpart show -p lists it as (a gpt context, not MBR): >>=20 >> 534528 7340032 da0p2 freebsd-swap (3.5G) >>=20 >> and gpart show -pl lists it as: >>=20 >> 534528 7340032 da0p2 Rock64swp2 (3.5G) >>=20 >> Okay noted on GPT not MBR method with gpart. >=20 > I did not happen to have a MBR example around. So I could > only show GPT. The note was more to avoid confusion than > anything, since the two are not equivalent for how they > work. >=20 >> By the way, what's the proper allocation size of swap in FreeBSD? >=20 > FreeBSD has a waring that it produces indicating possible mistuning > when you potentially have too much. An example is: I seem to have been on a mission to make some typos . . . "warning" would correct the above. > warning: total configured swap (2097152 pages) exceeds maximum = recommended amount (916632 pages). > warning: increase kern.maxswzone or reduce amount of swap. >=20 > The numbers are dependent on the amount of RAM present and > other details. >=20 > My understanding is that increasing kern.maxswzone has tradeoffs. > I avoid getting the message because I do not understand the > tradeoffs or how to manage the tradeoffs or even how to identify > an instance of hitting such a tradeoff. >=20 > For aarch64 I've been about to have swap of about 3.4 to 3.5 or > so times the amount of RAM without getting the warnings. "allowed", not "about". (This one was potentially confusing enough to justify this message.) > That > is why 3.5G in my RPi3B example. (So RAM+SWAP approx.=3D 4.5*RAM.) > (armv7 only allows more like 1.8 times the RAM before getting > the warning.) >=20 > I avoid even getting too close to the warning as there seems to > be some build-to-build variability in what fits vs. not. This > avoids having to frequently adjust the size. >=20 > Going from the other side, how much RAM+SWAP will your activities > use? To avoid accurately figuring out such, you may just want to > have near the 3.4 to 3.5 times RAM. (There have been times when > clang had memory use oddities that required more than normal for > a time, for example.) >=20 >> This RPi 3B has 1GB of RAM (~947 MB), do I need to set twice the = capacity of this physical RAM? >=20 > Ultimately your choice. How much parallel activity you > want to attempt likely contributes. If you build ports, > you might do so in a way that uses more RAM+SWAP than > system builds do, for example. >=20 >> (Note: swap file usage is subject to deadlock conditions >> avoided by use of swap partitions.) >>=20 >> This is noted. >>=20 >>=20 >> I use a serial console & ssh session only context to avoid >> having sizable competition for RAM. >>=20 >> I avoid using tmpfs because it competes for RAM use. >>=20 >> I use the likes of ( in, say, /boot/loader/conf ): >>=20 >> # >> # Delay when persistent low free RAM leads to >> # Out Of Memory killing of processes: >> vm.pageout_oom_seq=3D120 >>=20 >> This delays potential "killed: failed to reclaim memory" kills, >> possibly long enough to reach a state where sufficient memory is >> reclaimed. >>=20 >> Alright this is well noted too. >=20 > There is tuning related to "a thread waited too long to > allocate a page" that happens because of paging I/O > characteristics. But but I've not hit that type of > error. >=20 > I'll also note that the "out of swap space" case is a > misnomer in that it is one or two of 2 internal data > structures that is out of space, not necessarily the > swap space on the media. Again, I've not ever hit that > type of error. I'm not aware of tuning for this case. >=20 >> I'll note that the status "killed: failed to reclaim memory" does >> not require that swap be used much at all. Sustained low free RAM >> from just one process that always stays runnable and has a >> sufficiently large active set of pages can be sufficient to end up >> with such kills. Having swap allows for inactive pages to get out >> of the way, which can help. >>=20 >> I use the likes of ( in, say, /etc/ssyctl.conf ): >>=20 >> # >> # Together this pair avoids swapping out the process kernel stacks. >> # This avoids processes for interacting with the system from being >> # hung-up. >> vm.swap_enabled=3D0 >> vm.swap_idle_enabled=3D0 >>=20 >> This allows paging to the swap space but disallows moving >> kernel thread stacks to the swap space. Otherwise the >> processes used to interact with the RPi3 can become >> non-runnable, preventing such interactions. >>=20 >> Okay this too is well noted. >>=20 >>=20 >> I have NVMe or SSD based USB media, not microsd cards nor >> spinning rust. (I use just bootcode.bin and timeout files >> on microsd media for the RPi3B. Even the rest of the RPi* >> firmware is on the USB media, as well as u-boot.bin .) >=20 > This may contribute to why I've never gotten a "a thread > waited too long to allocate a page" on any system. (Some > systems, while bootable via USB3 media I have, also have > have even faster internal media that is normally used.) >=20 >> My usage of such a configuration struture for building >> software (world, kernel, ports) applies to all the >> systems I do such with, including ones with a lot more >> resources, including a lot more RAM. >>=20 >> Thanks for these inputs, noted on these things! I haven't tried NVMe = and SSD media in my RPi 3B. So, they are far more superior as compared = to microSD cards when it comes to building software? >=20 > My understanding is that microsd card media is fairly > generally not as good for such contexts: slower, fails > sooner, etc. >=20 > I happen to boot multiple types of machines from the > same media so I use USB3 media that is compatible with > USB2 use, a single such USB3 device not needing a > powered hub for use on the likes of an RPi3B. (Lots > of USB3 media around would require external power for > USB2 or an RPi3B use.) I need a powered hub for 2 or > more such media on a RPi3B. >=20 I'll note that the NVMe USB3 media seems to take longer than normal to power-up/reset. I ended up needing to have, for example, usb_pgood_delay assigned for u-boot.bin use in booting. The older USB3 SSD media did not require such, and still does not. This issue is not limited to booting RPi*'s in my context. (Although, where I use EDK2 UEFI/ACPI style booting, I've not had to do anything special for EDK2.) =3D=3D=3D Mark Millard marklmi at yahoo.com