Re: Restraining poudriere

On 6/12/21 4:16 PM, bob prohaska wrote:
> On Sat, Jun 12, 2021 at 01:26:16PM -0700, Jose Quinteiro wrote:
>> On 6/12/21 10:57 AM, bob prohaska wrote:
>>>
>>> Trying it now, hoping to see parallel core use.
>>
>> You won't. Setting PARALLEL_JOBS=1 means exactly one Poudriere worker
>> will run, and that make will not build in parallel. You have to decide
>> whether you want multiple Poudriere jobs, each using a single core at
>> most, one Poudriere job with make(1) potentially using several cores, or
>> some combination of both.
>>
>> The three variables that control this are:
>>
>> PARALLEL_JOBS, as you already know. This is the maximum number of
>> workers Poudriere will launch. It can be less than this if there are
>> multiple dependent packages waiting for a big package to build. This
>> happens a lot with Llvm and the like. Not having ALLOW_MAKE_JOBS set is
>> very frustrating in this case, because the big pig of a build will use
>> one core at most and take forever.
>>
>> ALLOW_MAKE_JOBS and MAKE_JOBS_NUMBER (make.conf). The first one allows
>> make(1) to launch parallel jobs, and the second controls the maximum
>> number of make jobs that will be launched. The problem here is that many
>> builds have lengthy steps that are not concurrent (I'm looking at you,
>> autotools!) A lot of times the actual compile and link steps take a tiny
>> fraction of the total build time.
>>
>> I chose to use a combination of both, and had to experiment with
>> different numbers for PARALLEL_JOBS mad MAKE_JOBS_NUMBER so that a
>> majority of my cores were used for the build, but I did not run out of
>> memory:
>> https://forums.FreeBSD.org/threads/poudriere-never-completes-first-run-without-errors.78000/post-486707
>>
> 
> It appears that starting poudriere with -J 1 and adding
> ALLOW_MAKE_JOBS=yes
> MAKE_JOBS_NUMBER=4
> to /etc/make.conf will come a little closer to making good use
> of the machine's puny resources. Have I got that right?
> 
This is very specific to the list of packages being built and the
machine used to build them, so you tell us!

> It also looks as if setting 
> USE_TMPFS=no
> in /usr/local/etc/poudriere.conf will help. Right now, with
> tmpfs enabled on a single thread the machine is using 800 MB
> of swap. 
> 
> I rather like your idea of setting a target load average. 
> Wish I knew how to do it...... methinks it isn't simple.
> 
The only way to do it now is by trial and error, and the runs are of
questionable reproducibility because I use ccache(1).

It would be super cool if Poudriere could be configured to target a load
average, and tweak MAKE_JOBS_NUMBER and PARALLEL_JOBS dynamically for
you. I suspect that's not simple to implement, and may not work very
well anyway. It's certainly beyond my abilities.

> Thanks again!
> 
> bob prohaska
> 
You're welcome!
Jose