Re: Too many pythons in poudriere

On Thu, Jul 08, 2021 at 09:41:13AM -0700, Mark Millard wrote:
> 
> 
> On 2021-Jul-8, at 08:45, bob prohaska <fbsd at www.zefox.net> wrote:
> 
> > Even with -J1 and no ALLOW_MAKE_JOBS I'm still
> > seeing five pythons occupying at least 3 GB on
> > the loose.
> 
> Actually I just looked and saw:
> 
> Swapinfo 7.36%
> 
> (Unlike the 83% or so I saw somewhat around 3 hours ago.)
> 
> Load Averages (220%) 2.20 2.18 1.76
> 
> Elapsed 12:54:56
> 
> I do not see a swaplog in http://www.zefox.org/~bob/swaplogs/
> to look at. So I can not see how much the peak swap space
> usage was so far (approximately).

Started a new swaplog:
http://www.zefox.org/~bob/swaplogs/202107182930.log

It came within a whisker of running out of swap, then abruptly
the python threads vanished and the build seems to be proceeding.
I'm curious if this was blind luck, or some adaptive behavior
by poudrirere. One other oddity: occasionally one see in top a
PID using more than 100% WCPU. Is one thread occupying two cores?

> 
> > I'm fairly sure this didn't happen
> > when using make by itself (IIRC it was -j2).
> > I also got rid of the mistaken directive in
> > poudriere.d/make.conf.
> 
> When I look at http://www.zefox.org/~bob/poudriere.d/make.conf
> now I see:
> 
> ALLOW_MAKE_JOBS=yes
> #MAKE_JOBS_NUMBER=2
> #.if ${.CURDIR:M*www/chromium}
> #MAKE_JOBS_NUMBER_LIMIT=2
> #.endif
> #.if ${.CURDIR:M*databases/sqlite3}
> #MAKE_JOBS_NUMBER_LIMIT=2
> #.endif
> #.if ${.CURDIR:M*www/firefox}
> #MAKE_JOBS_NUMBER_LIMIT=2
> #.endif
> 
> which does not match your wording.
> 

Thank you for catching my error. _now_ it's fixed.

[snip]
> To see what is getting CPU time that leads to
> the load averages being around 2 might take
> using something like top sorted by cpu time
> and watching for a while.
> 
> > There is a 
> > #MAX_MEMORY=8
> > in poudriere.conf, presumably GB.
> 
> Documented as GiB:
>
> # How much memory to limit jail processes to for *each builder*
> # in GiB (default: none)
> #MAX_MEMORY=8
> 
> Per builder, not per-make-process.
> Within a builder each make-process shares
> that size space with the others.
> 
> > That
> > looks like a good knob to play with. Would
> > setting it to something like 3 or 4 help?
> 
> If the memory use exceeds what you set, the builder
> process is likely killed. 
[snip] 

I was hopeful it might inhibit starting new PIDs
when memory/swap is below some threshold. Guess not. 

Thanks for writing!

bob prohaska