Re: We need to do something about build times

Tomoaki AOKI wrote on 2023/10/29 20:51:
> On Sat, 28 Oct 2023 16:10:11 +0200
> Moin Rahman <bofh@freebsd.org> wrote:
>>
>> Maybe there are still some place of improvements in poudriere's change detection mechanism specially BUILD_DEPENDS. :P
>>
> 
> And maybe indirect dependencies, too. ;-(
> 
> At worst, BUILD_DEPENDS should be recorded in pkg database and some
> option to retrieve them, like %r and %d.
> I strongly suspect the lack of this functionality affects mis-behaviour
> of poudriere on deciding build order and combination.
> 

I don't see any problem in detecting dependencies by poudriere.
The various other *_DEPENDS that have been added are only conditional BUILD_DEPENDS or RUN_DEPENDS.
For example,
LIB_DEPENDS: detect using libname and set both BUILD_DEPENDS and RUN_DEPENDS.
FETCH_DEPENDS: BUILD_DEPENDS used when distfiles are missing

I think this problem is more of a RUN_DEPENDS problem.
It is that ports that waste time when rebuilt frequently have been included in the following structure.

[updated popular port like glib] <--+-- run-depend -- [ zero or some port ] <-- run-depend -- [ commonly used in build-time dependencies like doxygen ] <--+
                                    |                                                                                                                      |
                                    |                                                                                                                      +-- build-depend --+
                                    |                                                                                                                                         |
                                    +--------------------------------------------------------------------------------------------------------------------------- run-depend --+-- [ port to be rebuilt as it lost popular port ]

It is very confusing :)
It seems that any dependencies, whether build or run, will be included in this graph, and that any ports where the run dependencies is broken will be rebuilt.

In this regard, poudriere already has a method for pseudo-testing.
As I have already written previously, it will be as follows

poudriere bulk -j ... -C updated/upstream-popular-port affected-by-upstream-update/port-to-queue

Here are some examples... But I can't think of a good example right off the bat :)
First, there is a recent common

poudriere bulk -j ... -n -C graphics/vulkan-headers devel/py-qt5-pyqt

Never forget the -n option when this is performed. Otherwise, disaster will befall the already built package :)
This reproduces vulkan updates. This will delete a lot of qt5-*.
However, as you can see from the terminal logs, only qt5-gui is rebuilt, avoiding the loss of qt5-webkit.

Another.

poudriere bulk -j ... -n -C multimedia/aom multimedia/gstreamer1-plugins-core

aom has also been updated rather frequently.
However, while rebuilding ffmpeg is unavoidable, rebuilding that much will avoid rebuilding gstreamer1 and the ports that use it.

poudriere already has the option -S to not detect dependencies.
But this is too much. It also makes it difficult to find packages whose dependencies have been cut off.
What should be done in poudriere to make this problem smaller is not a review of dependency detection methods.
The decision to delete the package should be delayed.

When a copy (hardlink) is made in the .building directory, unwanted packages are removed by not being processed.
It would be nice to have the ability to stop doing that, copy all packages and then re-examine them to see if they need to be rebuilt when they are rebuilt.

It doesn't sound too easy...

Regards.