Re: This is going to break port building without poudriere!

Am 2024-01-25 20:57, schrieb Luca Pizzamiglio:

> Hi Stefan,
> 
> I did reply to your first email, but not to your second one.
> I preferred (in agreement with portmgr@) to open the discussion with 
> everyone, instead of keeping it just between you and me.
> 
> As you can read in the email you have linked, I didn't ignore your 
> comments.
> 
>> This implementation will break port dependencies, since there is no 
>> way
>> a port can depend on a specific sub-package - there even is no way a
>> non-default sub-package can be built without manual selection of the
>> options that activate its creation.
> 
> A port can depend on a specific sub-package. The category/origin~subpkg 
> is the chosen format.
> Non-default subpackages shouldn't exist. In the aforementioned email, I 
> explicitly say that IF a subpackage is enabled by an option, the option 
> must be enabled by default.
> Your previous comments contributed to making this point clearer.
> 
>> Dependencies stated in the port Makefile are converted into package
>> dependencies that can be resolved by the "pkg" command, but that 
>> cannot
>> be directly used to build and install the requested sub-package from
>> a port.
> 
> I don't know where you get this idea, but it's not how it works.
> If a port has a subpackage dependency, the category/origin~subpkg is 
> the chosen format to express that dependency.
> The related port (category/origin) has to be built and installed.
> By running `make install`, the whole port is installed, subpackages as 
> well.
> The only "issue" I see is that via 'make install' you cannot install 
> only the subpackage, but the entire port only.
> However this is not different than before.

I think Stefans expectations about such a feature are different from the 
understand of the implementor of the feature in our tree. Somewhat a 
clash of an idealistic view and reality.

To me (and I assume to Stefan too) it looks like slave ports will go 
away and subpackages will be used instead. A slave port may only compile 
a subset, and a subpackages aware port will compile everything 
(simplified view, not true where slave ports exclude a feature instead 
of excluding a file). From this point of view, a port can not depend on 
a subpackage in the sense of a port can not depend only on a subset of 
the php extensions included in the main php build (if/when it is 
converted to subpackages). As such a build from ports will have all php 
extensions included in the php port installed, whereas a pkp based 
install can limit the amount of installed php extensions to what is 
required. At least this is what I understand based upon what I have read 
about subpackages. As the documentation is not ready and I haven't 
looked at the code, this understanding may off course be wrong.

>> Ports that do not create sub-packages can still be depended on by 
>> other
>> ports, but as critical dependencies have been depended to sub-packages
>> a ever large fraction of ports will only build in poudriere.
> 
> Again, no.
> If you run make install on any port, it will be built. The package and 
> the subpackages are all installed.
> You cannot install a portion of a port, and it has never been possible.

Is my understanding correct that subpackages aware ports are supposed to 
replace master/slave ports? In the sense of slave ports will be deleted 
once a port is converted to a subpackages aware port (except where slave 
ports exclude features from binaries like git-lite... yes, git-lite is 
implemented as a flavour and as such we cover this in a different way 
and such slave ports if they still exist can maybe be converted to use 
flavours)? I assume yes to both questions.  With this assumption:

With master/slave ports this is possible. And replacing master/slave 
ports with a subpackages aware port will remove this possibility.

> Nothing is broken here and I fail to understand where you get the idea 
> of this behavior.
> I wrote tests and examples to implement the feature and get them 
> working before adding support to poudriere.
> 
>> This change does also obviously break port management tools like 
>> portmaster,
>> which took me significant effort to adapt to FLAVOR support (which 
>> also had
>> been implemented without consideration for other tools than 
>> poudriere), and
>> which I have been maintaining since then.
> 
> Yes, portmaster need to be able to parse the new dependencies format, 
> by removing the ~subpkg. By removing the suffix, you get the 
> category/port and everything is fine.
> As announced, we are keeping subpackages adoption blocked with a 
> git-hook to give time to maintainers to add support to subpkg and to 
> introduce the feature slowly.
> portmgr@ doesn't officially support neither portmaster nor portupgrade. 
> We simply lack the manpower to do it.
> 
>> The reason given for sub-packages support is reduced build time for 
>> some
>> packages that share a common distfile (e.g. qt5) when building 
>> official
>> packages.
> 
> As explained, in the short term there are this benefit and getting rid 
> of many master/slave ports (hard to maintain)
> But there are several use cases in the future.
> For instance, work is in place to provide debug symbols as subpackage, 
> in a similar way as pkg-base.
> 
>> But this comes at a high cost for all builds outside the package build
>> cluster, since now lots of unnecessary sub-trees will be compiled and
>> installed, if only one program (i.e. sub-package) is desired.
> 
> This is the reason why OPTIONS with SUBPACAKGES have been introduced, 
> to reduce build time for port builders.

The complaint/fear of Stefan is that there does not seem to be a way to 
e.g.:
  1) have php converted to a subpackages aware port which will remove 
some of the php extensions ports as they will be handled as subpackages 
of the lang/php port
  2) depend upon one of the extensions which are now integrated as a 
subpackage of the php port but do not depend upon all extensions which 
are available as subpackages
  3) as a result have all php extensions build and installed (and as such 
loaded at run time automatically), even those which are not needed, when 
installing via ports

I think what Stefan is asking for, is to have a way that only build a 
subpackage if desired (default behavior = build everything, but the 
possibility to only build the subpackage specific part in case the 
upstream source allows this, like in the php example).

>> You are breaking use-cases of a large number of users that still build
>> ports without poudriere. This is especially causing a barrier to entry
>> of new port developers, since this will require them to setup 
>> poudriere
>> before they can begin port development. (This will only become an 
>> issue
>> over time, as more and more dependencies will have been converted to
>> sub-packages, but then there will be no way back.)
> 
> I still fail to see what is going to break. Non-poudriere users can 
> experience longer build time for dependencies, but dependencies can be 
> installed via packages, at least most of the time.

What is broken is that you _have_ to install such dependencies via 
packages in this case if you want to have the minimal install. With 
master/slave ports this is not required, the user is free to install 
everything as a port. "cd 
/usr/ports/port/which/depends/upon/a/subset/of/the/php/extensions; make 
install" will be different than "pkg install 
port/which/depends/upon/a/subset/of/the/php/extensions". And in this 
example this can also have a security implication (php stuff may be 
exposed to the internet and more extensions which can have a security 
issue may as such be exposed when installed as a port but not when 
installed as a package).

Bye,
Alexander.

-- 
http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.org    netchild@FreeBSD.org  : PGP 0x8F31830F9F2772BF