Re: FreeBSD ports community is broken [port building configuration notes]

[poudriere bulk status update.]

On Mar 5, 2024, at 18:43, Mark Millard <marklmi@yahoo.com> wrote:

> [I noticed that my SWAP figures were not self consistent for the armv7.]
> 
> On Feb 18, 2024, at 09:50, Mark Millard <marklmi@yahoo.com> wrote:
> 
>> [I also forgot to mention an important FreeBSD configuration setting
>> as well. It is not specific to poudriere use.]
>> 
>>> On Feb 18, 2024, at 09:13, Mark Millard <marklmi@yahoo.com> wrote:
>>> 
>>> [I forgot to mention the armv7 core count involved: 4]
>>> 
>>> On Feb 18, 2024, at 08:52, Mark Millard <marklmi@yahoo.com> wrote:
>>> 
>>>> Aryeh Friedman <aryehfriedman_at_gmail.com> wrote on
>>>> Date: Sun, 18 Feb 2024 10:37:06 UTC :
>>>> 
>>>>> It should not require
>>>>> prodiere running on a supermassive machine to work (in many cases
>>>>> portmaster and make install recursion fail where prodiere works).
>>>> 
>>>> As for configuring for small, slow systems relative to
>>>> resource use, I provide some settings that I've
>>>> historically used below. Then I have some other notes
>>>> after that material.
>>>> 
>>>> For a 2 GiByte RAM armv7 system with 3 GiByte swap space
>>>> and a UFS file system, no use of tmpfs in normal operation
>>>> (since it competes for RAM+SWAP generally):
> 
> Actually: 2 GiByte RAM armv7 has 3.6 GiByte SWAP space, with
> some margin. Ever so slightly over 3.8 GiBytes got the mistuning
> warning but there is variability across builds so I try to avoid
> repeated adjustments by picking somewhat smaller.
> 
>>> FYI: The armv7 has 4 cores.
>>> 
>>>> /usr/local/etc/poudriere.conf has . . .
>>>> 
>>>> NO_ZFS=yes
>>>> USE_TMPFS=no
>>>> PARALLEL_JOBS=2
>>>> ALLOW_MAKE_JOBS=yes
>>>> MAX_EXECUTION_TIME=432000
>>>> NOHANG_TIME=432000
>>>> MAX_EXECUTION_TIME_EXTRACT=14400
>>>> MAX_EXECUTION_TIME_INSTALL=14400
>>>> MAX_EXECUTION_TIME_PACKAGE=57600
>>>> MAX_EXECUTION_TIME_DEINSTALL=14400
>>>> 
>>>> /usr/local/etc/poudriere.d/make.conf has . . .
>>>> 
>>>> MAKE_JOBS_NUMBER=2
>>>> 
>>>> /etc/fstab does not specify any tmpfs use or the
>>>> like: avoids competing for RAM+SWAP.
>>>> 
>>>> The 3 GiBytes of swap space is deliberate: RAM+SWAP
>>>> is important for all means of building in such a
>>>> context: there are a bunch of ports that have
>>>> large memory use for building in all cases.
>>>> 
>>>> [armv7 allows around RAM+SWAP=2.5*RAM before
> 
> That equation should have been RAM+SWAP==2.8*RAM
> (with margin considered), so SWAP==1.8*RAM. (With
> a small enough RAM 2.7*RAM might need to be used,
> for example.)
> 
> So the 2 GiByte RAM leads to a 5.6 GiByte RAM+SWAP
> for the builders and other uses to share.
> 
> I may set up a modern experiment to see if the
> combination:
> 
> PARALLEL_JOBS=2
> ALLOW_MAKE_JOBS=yes (with MAKE_JOBS_NUMBER=2)
> 
> still completes for a build that would end up with
> llvm18 and rust likely building in parallel for
> much of the time (if it completed okay, anyway).
> Something like 265 ports would be queued, the last
> few of which include some use of llvm18 and of
> rust.
> 
> If it failed, I'd revert to using PARALLEL_JOBS=1
> and MAKE_JOBS_NUMBER=3 and see how that went.
> llvm18 and rust would no longer build in parallel.
> If that also failed, I'd revert to
> MAKE_JOBS_NUMBER=2 . The last option would be
> MAKE_JOBS_NUMBER=1 .
> 
> It has been a notable time since I last did such
> an exploration on such a small configuration.
> 
>>>> tradeoff/mistuning notices are generated. aarch64
>>>> and amd64 allow more like RAM+SWAP=3.4*RAM before
>>>> such notices are reported. The detailed multiplier
>>>> changes some from build to build, so I leave
>>>> margin in my figures to avoid the notices.]
>>>> 
>>>> I also historically use USB SSD/NVMe media, no
>>>> spinning rust, no microsd cards or such.
>> 
>> /boot/loader.conf has . . .
>> 
>> #
>> # Delay when persistent low free RAM leads to
>> # Out Of Memory killing of processes:
>> vm.pageout_oom_seq=120
>> 
>> This is important to allowing various things
>> to complete. (The default is 12. 120 is not
>> the maximum but has been appropriate in my
>> context. The figure is not in time units but
>> larger increases the observed delay so more
>> work gets done before OOM activity starts.)
>> 
>> Using vm.pageout_oom_seq is not specific to
>> poudriere use.
>> 
>>>> As far as more ports building in poudriere than in
>>>> "portmaster and make install recursion" in other
>>>> respects than resources: it is easier to make ports
>>>> build in poudriere. It provides the simpler/cleaner
>>>> context for the individual builders. More things
>>>> lead to failure outside poudriere that are just not
>>>> issues when poudriere is used so more care is needed
>>>> setting up the ports for the likes of portmaster use.
>>>> (And, yes, I used to use portmaster.) The required
>>>> range of testing contexts is wider for use of the
>>>> likes of portmaster to know that the port build will
>>>> just work in the full range of contexts.
>>>> 
>>>> Such issues adds to the port maintainer/committer
>>>> development burdens when portmaster or the like are
>>>> the target level/type of support.
>>>> 
>>>> (Note: synth may be more like poudriere for this
>>>> but I've historically had use of platforms that
>>>> synth did not support and so have not looked into
>>>> the details.)

Context: 1GHz, 4 core, cortex-a7 (armv7), 2 GiBytes RAM, USB2.
RAM+SWAP: 5.6 GiBytes. Also, this is doing my normal armv7 (and
aarch64) style of devel/llvm* build: OPTION'd to BE_NATIVE
instead of BE_STANDARD and OPTION'd to not build MLIR.

The poudriere bulk has finished llvm18 and rust, having built
206 packages with 59 to go. Summarizing via package builds that
took over an hour:

[01:51:31] [01] [01:00:07] Finished lang/perl5.36 | perl5-5.36.3_1: Success
[08:55:35] [02] [03:08:09] Finished devel/icu | icu-74.2,1: Success
[13:17:38] [02] [01:28:32] Finished lang/ruby31 | ruby-3.1.4_1,1: Success
[14:17:44] [01] [09:20:55] Finished devel/cmake-core | cmake-core-3.28.3: Success
[4D:01:03:43] [02] [3D:08:48:53] Finished lang/rust | rust-1.76.0: Success
[4D:06:26:24] [02] [03:09:35] Finished devel/binutils@native | binutils-2.40_5,1: Success
[4D:14:54:31] [02] [03:38:55] Finished devel/aarch64-none-elf-gcc | aarch64-none-elf-gcc-11.3.0_3: Success
[4D:16:13:00] [01] [4D:01:55:03] Finished devel/llvm18@default | llvm18-18.1.0.r3: Success

So: llvm18 started before rust and finished after rust, each
mostly using 2 hardware threads. About the last 1.5 hr for
llvm18 was llvm18 being packaged, after somewhat over 96 hours
of mostly 2 hardware threads working on it. The vast majority
of the build time was for the build phase.

I have a modified top that monitors and reports some "MAXimum
OBServed" figures (MaxObsYYY figures). As of llvm18 finishing,
that top was reporting:

2794Mi MaxObs(Act+Wir+Lndry+SwapUsed)
(Inact can be an arbitrary mix of dirty and clean pages and,
so, is not included.)

Swap: 995524Ki MaxObsUsed

Thus, it used up to around half of the RAM+SWAP to get that
far. (Rust and llvm18's peak RAM+SWAP usages need not have
been over the same time period. But there was RAM+SWAP room
for a larger overall peak.)

[Note: The peak RAM+SWAP use was during a period of llvm18's
build running various llvm-tblgen examples.]

As stands, it looks like the poudriere bulk run will complete
just fine for the configuration that I specified, with margin
for variations in peak RAM+SWAP usage.


===
Mark Millard
marklmi at yahoo.com