Re: Too many pythons in poudriere

On 2021-Jul-9, at 16:07, bob prohaska <fbsd at www.zefox.net> wrote:

> On Fri, Jul 09, 2021 at 12:46:47PM -0700, Mark Millard wrote:
>> 
>> Going in different direction, when you updated:
>> 
>> /usr/local/poudriere/poudriere-system
>> 
>> to have Jail OSVERSION: 1400024 (and probably
>> matching the host OS main-n247590-5dd84e315a9),
>> matching /usr/src/ as well, did you rebuild all
>> your ports under the coherent combination?
>> 
> No.
>> Or are you still using port builds in poudriere that
>> were built with the jail OSVERSION being older than
>> the /usr/src/ the builds were based on?
>> 
> 
> Yes.
> 
>> Just like devel/llvm10 had a command that only-sometimes
>> had problems when built via the incoherent combination,
>> other ports could have odd behavior --but only sometimes.
>> 
>> My guess is that you have not done such because it would
>> have taken far more time than I've noticed in your
>> activities.
>> 
> Correct!
> 
>> I recommended before doing such a rebuild of the ports
>> and then a reinstall of them. Given the odd behaviors
>> being observed that do not repeat each time tried, I
>> still make the recommendation.
>> 
> 
> Just to be clear, after updating kernel and world I gather the 
> suggestion is to repeat
> 
> # cd /usr/src
> # make installworld DESTDIR=/usr/local/poudriere/poudriere-system DB_FROM_SRC=1
> # make distrib-dirs DESTDIR=/usr/local/poudriere/poudriere-system DB_FROM_SRC=1
> # make distribution DESTDIR=/usr/local/poudriere/poudriere-system DB_FROM_SRC=1

I though you had already synchronized /usr/local/poudriere/poudriere-system
to match /usr/src/ and the host root file system in use. If so, no reason
to repeat until /usr/src/ (and the booted host root file system) is updated
for some reason. Avoid mixes where /usr/src/ is mismatched with the root
file system at the time.

Of course, you might want to do another update to those for all I know.

One thing you did not show was the command for after the above block of
4 commands for updates:

# make -DBATCH_DELETE_OLD_FILES delete-old DESTDIR=/usr/local/poudriere/poudriere-system DB_FROM_SRC=1

If you have not done that yet, it would be appropriate.

For the current context of starting over, another command that
is appropriate (that would normally be delayed to after all the
poudreire bulk commands are done [were prior package builds
were instead being put to use]) is:

# make -DBATCH_DELETE_OLD_FILES delete-old-libs DESTDIR=/usr/local/poudriere/poudriere-system DB_FROM_SRC=1

I'll note that the host updating procedure also involves using
delete-old and, after package updates are all installed,
delete-old-libs . (No explicit DESTDIR= or DB_FROM_SRC= use
involved for the root file system update.)

> Next, 
> cd /usr/local/poudriere
> poudriere jail -d main -C all
> to get rid of the old jail and packages, then re-make the jail with 
> 
> # poudriere ports -c -m null -M /usr/ports
> # poudriere jail -c -j main -m null -M /usr/local/poudriere/poudriere-system -S /usr/src -v 14.0-CURRENT
> 
> Have I got this right? One thing I'm hesitant about is 
> the -C all option. 

No reason to delete and recreate the jail, just do a bulk build that
rebuilds the ports. The first "poudriere bulk" of a sequence can use
the "-c" option to clear out the existing packages (and logs) before
the builders start building.

As far as I know you eventually want to install:

# pkg install chromium

(that will also install things required to run chromium).

I'm not sure if you have a list of other packages that
you want explicitly installed. So my notes below need
not cover everything that you want to do.

(I do not know if ALLOW_MAKE_JOBS= and such is appropriate for the
llvm10 bulk build but it may fit in the resources okay:)

# poudriere bulk -j main -c devel/llvm10 > bulk_from_scratch_for_llvm10_1st.log

(I'm unclear if you would want a -J? to limit the builders
count for the above command: if so, use it too.)

That -c will first clear out the existing package builds (and
the logs according to the documentation). Then it will start
from scratch building the required ports.

This will build a lot of ports, those needed to in turn build
devel/llvm10 . So having -J1 used and ALLOW_MAKE_JOBS missing
would make for a very long build time. You probably want to
allow some combination of multiple cores to be in use,
possibly also involving MAKE_JOBS_NUMBER= use. You know more
about what combinations worked before.

Reminder that ALLOW_MAKE_JOBS= control would go in:

/usr/local/etc/poudriere.conf

but MAKE_JOBS_NUMBER= control would go in:

/usr/local/etc/poudriere.d/make.conf

(I do not know if ALLOW_MAKE_JOBS= and such is appropriate for the
rust bulk build but some combination may fit in the resources okay:)

# poudriere bulk -j main lang/rust > bulk_for_rust_2nd.log

(I'm unclear if you would want a -J? to limit the builders
count for the above command: if so, use it too. You know
more about what combinations worked before.)

(Without ALLOW_MAKE_JOBS= is likely appropriate --or at least the
MAKE_JOBS_NUMBER= may be needed-- for:)

# poudriere bulk -j main www/chromium > bulk_for_chromium_3rd.log

(I'm unclear if you would want a -J? to limit the builders
count for the above command: if so, use it too. You know
more about what combinations worked before.)

Note that the management of the limited RPi3B context involves
adjusting things like ALLOW_MAKE_JOBS= and -J? usage for each
huge build to limit resource use to an appropriate scale for
those builds. For smaller builds, more can be allowed (more
adjustments).

I do not know if there are other ports that you would want
to build and later install that are not automatically built
by the above or if you have a list of such in a file that
can be used with -f . In your context this would probably
be a list of ports that do not involve huge builds, given
that you had already done the above bulk runs (and any others
that might involve more huge builds).

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)