Re: This is going to break port building without poudriere! (was: Subpackage explanations)

Hi Stefan,

I did reply to your first email, but not to your second one.
I preferred (in agreement with portmgr@) to open the discussion with
everyone, instead of keeping it just between you and me.

As you can read in the email you have linked, I didn't ignore your comments.

This implementation will break port dependencies, since there is no way
> a port can depend on a specific sub-package - there even is no way a
> non-default sub-package can be built without manual selection of the
> options that activate its creation.
>
A port can depend on a specific sub-package. The category/origin~subpkg is
the chosen format.
Non-default subpackages shouldn't exist. In the aforementioned email, I
explicitly say that IF a subpackage is enabled by an option, the option
must be enabled by default.
Your previous comments contributed to making this point clearer.


> Dependencies stated in the port Makefile are converted into package
> dependencies that can be resolved by the "pkg" command, but that cannot
> be directly used to build and install the requested sub-package from
> a port.
>
I don't know where you get this idea, but it's not how it works.
If a port has a subpackage dependency, the category/origin~subpkg is the
chosen format to express that dependency.
The related port (category/origin) has to be built and installed.
By running `make install`, the whole port is installed, subpackages as well.
The only "issue" I see is that via 'make install' you cannot install only
the subpackage, but the entire port only.
However this is not different than before.

Has there been a general consensus that support for direct port building
> (without poudriere) will be abandoned?
>
Port building is not abandoned, and it's actively supported, as it's the
foundation of poudriere.
If a regression has been introduced, by me or anyone else, it has to be
fixed.
I may have overlooked some use cases, but AFAIK I didn't introduce any
regression (confirmed by the exp-run)


> Ports that do not create sub-packages can still be depended on by other
> ports, but as critical dependencies have been depended to sub-packages
> a ever large fraction of ports will only build in poudriere.
>
Again, no.
If you run make install on any port, it will be built. The package and the
subpackages are all installed.
You cannot install a portion of a port, and it has never been possible.
Nothing is broken here and I fail to understand where you get the idea of
this behavior.
I wrote tests and examples to implement the feature and get them working
before adding support to poudriere.

This change does also obviously break port management tools like portmaster,
> which took me significant effort to adapt to FLAVOR support (which also had
> been implemented without consideration for other tools than poudriere), and
> which I have been maintaining since then.
>
Yes, portmaster need to be able to parse the new dependencies format, by
removing the ~subpkg. By removing the suffix, you get the category/port and
everything is fine.
As announced, we are keeping subpackages adoption blocked with a git-hook
to give time to maintainers to add support to subpkg and to introduce the
feature slowly.
portmgr@ doesn't officially support neither portmaster nor portupgrade. We
simply lack the manpower to do it.

The reason given for sub-packages support is reduced build time for some
> packages that share a common distfile (e.g. qt5) when building official
> packages.
>
As explained, in the short term there are this benefit and getting rid of
many master/slave ports (hard to maintain)
But there are several use cases in the future.
For instance, work is in place to provide debug symbols as subpackage, in a
similar way as pkg-base.


> But this comes at a high cost for all builds outside the package build
> cluster, since now lots of unnecessary sub-trees will be compiled and
> installed, if only one program (i.e. sub-package) is desired.
>
This is the reason why OPTIONS with SUBPACAKGES have been introduced, to
reduce build time for port builders.


> You are pessimizing the build for thousands of users to spare a few
> cycles for a very small percentage of packages built once in a while on
> the build cluster.
>
I have already explained this point above.

I had pointed out other issues with this approach in the review comment
> and in the private mail I sent 2024-01-02 after finding that this version
> has been committed without formal acceptance of your review D40549.
>
As I replied to you already, there has been no formal acceptance in
phabricator, but there was consensus in portmgr@ to land it.
I apologize for not having used the appropriate reviews channel, I totally
agree that it has not been a good behavior from my side, as I'm not
providing a good example.

You are breaking use-cases of a large number of users that still build
> ports without poudriere. This is especially causing a barrier to entry
> of new port developers, since this will require them to setup poudriere
> before they can begin port development. (This will only become an issue
> over time, as more and more dependencies will have been converted to
> sub-packages, but then there will be no way back.)
>
I still fail to see what is going to break.
Non-poudriere users can experience longer build time for dependencies, but
dependencies can be installed via packages, at least most of the time.

Your announcement mentions some of the issues, but does not offer any
> actual solution.
>

A sane implementation of sub-packages would not make their creation
> depend on OPTION settings, with no way to determine the required OPTION
> setting for a non-default sub-package (i.e., only default sub-packages
> can be depended on).
>
> I'm not going to repeat all the issues pointed out in
>
>         https://reviews.freebsd.org/D16457#715443
>
> but had on multiple occasions pointed out that a sane mechanism would
> start with "SUB_PACKAGE_DEFINE" and "SUB_PACKAGE_DEFAULT" variables and
> with a mechanism to determine the required OPTION values from the actual
> set of sub-packages to be built (instead of the opposite, to make the
> selection of sub-packages depend on the OPTIONs).
>
I clearly addressed this topic in the aforementioned email.
We chose not to follow this approach, and I motivated it (changing
configuration of other ports it's NOT something to have IMO).
We clearly disagree on this point.
I have opened up the discussion to get feedback from the rest of the
community as well.
So far, you are the only one strongly against this approach.

Best regards,
pizzamig