Re: FreeBSD ports community is broken [port building configuration notes]

[I noticed that my SWAP figures were not self consistent for the armv7.]

On Feb 18, 2024, at 09:50, Mark Millard <marklmi@yahoo.com> wrote:

> [I also forgot to mention an important FreeBSD configuration setting
> as well. It is not specific to poudriere use.]
> 
>> On Feb 18, 2024, at 09:13, Mark Millard <marklmi@yahoo.com> wrote:
>> 
>> [I forgot to mention the armv7 core count involved: 4]
>> 
>> On Feb 18, 2024, at 08:52, Mark Millard <marklmi@yahoo.com> wrote:
>> 
>>> Aryeh Friedman <aryehfriedman_at_gmail.com> wrote on
>>> Date: Sun, 18 Feb 2024 10:37:06 UTC :
>>> 
>>>> It should not require
>>>> prodiere running on a supermassive machine to work (in many cases
>>>> portmaster and make install recursion fail where prodiere works).
>>> 
>>> As for configuring for small, slow systems relative to
>>> resource use, I provide some settings that I've
>>> historically used below. Then I have some other notes
>>> after that material.
>>> 
>>> For a 2 GiByte RAM armv7 system with 3 GiByte swap space
>>> and a UFS file system, no use of tmpfs in normal operation
>>> (since it competes for RAM+SWAP generally):

Actually: 2 GiByte RAM armv7 has 3.6 GiByte SWAP space, with
some margin. Ever so slightly over 3.8 GiBytes got the mistuning
warning but there is variability across builds so I try to avoid
repeated adjustments by picking somewhat smaller.

>> FYI: The armv7 has 4 cores.
>> 
>>> /usr/local/etc/poudriere.conf has . . .
>>> 
>>> NO_ZFS=yes
>>> USE_TMPFS=no
>>> PARALLEL_JOBS=2
>>> ALLOW_MAKE_JOBS=yes
>>> MAX_EXECUTION_TIME=432000
>>> NOHANG_TIME=432000
>>> MAX_EXECUTION_TIME_EXTRACT=14400
>>> MAX_EXECUTION_TIME_INSTALL=14400
>>> MAX_EXECUTION_TIME_PACKAGE=57600
>>> MAX_EXECUTION_TIME_DEINSTALL=14400
>>> 
>>> /usr/local/etc/poudriere.d/make.conf has . . .
>>> 
>>> MAKE_JOBS_NUMBER=2
>>> 
>>> /etc/fstab does not specify any tmpfs use or the
>>> like: avoids competing for RAM+SWAP.
>>> 
>>> The 3 GiBytes of swap space is deliberate: RAM+SWAP
>>> is important for all means of building in such a
>>> context: there are a bunch of ports that have
>>> large memory use for building in all cases.
>>> 
>>> [armv7 allows around RAM+SWAP=2.5*RAM before

That equation should have been RAM+SWAP==2.8*RAM
(with margin considered), so SWAP==1.8*RAM. (With
a small enough RAM 2.7*RAM might need to be used,
for example.)

So the 2 GiByte RAM leads to a 5.6 GiByte RAM+SWAP
for the builders and other uses to share.

I may set up a modern experiment to see if the
combination:

PARALLEL_JOBS=2
ALLOW_MAKE_JOBS=yes (with MAKE_JOBS_NUMBER=2)

still completes for a build that would end up with
llvm18 and rust likely building in parallel for
much of the time (if it completed okay, anyway).
Something like 265 ports would be queued, the last
few of which include some use of llvm18 and of
rust.

If it failed, I'd revert to using PARALLEL_JOBS=1
and MAKE_JOBS_NUMBER=3 and see how that went.
llvm18 and rust would no longer build in parallel.
If that also failed, I'd revert to
MAKE_JOBS_NUMBER=2 . The last option would be
MAKE_JOBS_NUMBER=1 .

It has been a notable time since I last did such
an exploration on such a small configuration.

>>> tradeoff/mistuning notices are generated. aarch64
>>> and amd64 allow more like RAM+SWAP=3.4*RAM before
>>> such notices are reported. The detailed multiplier
>>> changes some from build to build, so I leave
>>> margin in my figures to avoid the notices.]
>>> 
>>> I also historically use USB SSD/NVMe media, no
>>> spinning rust, no microsd cards or such.
> 
> /boot/loader.conf has . . .
> 
> #
> # Delay when persistent low free RAM leads to
> # Out Of Memory killing of processes:
> vm.pageout_oom_seq=120
> 
> This is important to allowing various things
> to complete. (The default is 12. 120 is not
> the maximum but has been appropriate in my
> context. The figure is not in time units but
> larger increases the observed delay so more
> work gets done before OOM activity starts.)
> 
> Using vm.pageout_oom_seq is not specific to
> poudriere use.
> 
>>> As far as more ports building in poudriere than in
>>> "portmaster and make install recursion" in other
>>> respects than resources: it is easier to make ports
>>> build in poudriere. It provides the simpler/cleaner
>>> context for the individual builders. More things
>>> lead to failure outside poudriere that are just not
>>> issues when poudriere is used so more care is needed
>>> setting up the ports for the likes of portmaster use.
>>> (And, yes, I used to use portmaster.) The required
>>> range of testing contexts is wider for use of the
>>> likes of portmaster to know that the port build will
>>> just work in the full range of contexts.
>>> 
>>> Such issues adds to the port maintainer/committer
>>> development burdens when portmaster or the like are
>>> the target level/type of support.
>>> 
>>> (Note: synth may be more like poudriere for this
>>> but I've historically had use of platforms that
>>> synth did not support and so have not looked into
>>> the details.)



===
Mark Millard
marklmi at yahoo.com