Re: This is going to break port building without poudriere!
Date: Tue, 30 Jan 2024 10:30:59 UTC
Am 2024-01-30 10:42, schrieb Luca Pizzamiglio: > Hi Alexander. > > Subpackage modularization of existing ports (i.e. qt6-tools) provides > benefits to package users/builders: smaller dependency footprint, > smaller usage on disk (useful for embedded systems), smaller jails. > There are no benefits nor regressions for port builders for this > specific use case. Nicely worded. I agree to that. At the same time it is a regression for port builders. Installing from ports is not on par with installing from pkg anymore in this case. What you describe means we can use subpackages only for leaf ports. Ports which are a dependency can not converted to subpackages (in the sense of "no slave port available for the subpackages"). > On the other hand, moving master/slave ports to subpackages comes with > the cost of losing the ability to build the slave ports independently. > For package users there is no change and some benefit for package > builders. > The issue can be mitigated by introducing options to select which > subpackage to build (available for make install as well), but, still, a > single subpackage cannot be built independently. This is not a proper mitigation. I used mail/roundcube and lang/php as a specific example for this particular case where it is not a proper mitigation. > This limitation _can_ be unacceptable in some cases. For example, > bofh@, the php maintainer, considers subpackages not the way to go, so > the master/slave approach will stay. Do you see that you just told that one of the best role models for subpackages according to you previous mail will not use subpackages because of the limitations I highlighted? > We introduced subpackages in the framework to explore all use cases, by > trying and testing its adoption. > By doing so we have already identified some issues (i.e. USES is not > subpackage-aware yet) and interesting new use cases (subpackage with > debug symbols). You are surely aware that you haven't mentioned one of the points I talked about as an issue, don't you? I have not seen any technical answer which shows that my technical description of issues is wrong. I want to make you aware that I understand the responses from you and the others in this thread as: "we do not care about those regressions and do not want to fix them" (I understand it like that because I see no acknowledgement of those issues, just answers which look like a smokescreen and pointing into directions of good faith). If you would acknowledge the technical issues highlighted in this thread (or show evidence that those regressions can not happen) and tell that the adoption of subpackages is monitored to not introduce those regressions I talk about, my understanding of the situation would change. > As per master/slave merge use case, we will let the maintainers decide > if they want to move forward with subpackage adoptions, knowing the > regression for port builders, but we won't push them in this direction. I consider it unfortunate that you describe it like that. I was hoping you would tell that portmgr will prevent the removal of slave packages and enforce the rule of having slave ports for subpackages aware ports to prevent regressions for users which install from ports (until the technical valid issues I have pointed out are resolved -- and they can be fixed, a first step would be to make the names of subpackages match normal packages names = replacing the '~' in the name with a '-' ASAP to prevent churn later). Note, in src some big discussion like this of having several committers identify regressions and no immediate fix would lead to a backout until it is fixed. I do not ask for a backout of this, but I ask for a strong lock/policy by portmgr on the subpackages feature like I already described in my previous mails (until the regressions the use of subpackages would create are fixed). This would allow for more experimentation by a lot of people without the need to patch the ports tree and without introducing regressions for a simple "make install" of ports with a lot of dependencies. Bye, Alexander. > Best regards, > Luca > > On Mon, Jan 29, 2024 at 10:04 AM Alexander Leidinger > <Alexander@leidinger.net> wrote: > >> Am 2024-01-27 16:59, schrieb Luca Pizzamiglio: >> >>> Hi Stefan. >>> >>> Let's start from the beginning, as it seems that things are not >>> clear. >>> >>> Subpackages is a feature to create multiple packages from one build >>> Those subpackages can depend on the main package. >>> The main package cannot depend on any subpackages. >>> This limits the cases where subpackages can be applied. The main >>> package MUST be independent from its subpackages. Subpackages can >>> only >>> add features (like a plugin). >>> To recall your NLS example (ref >>> https://reviews.freebsd.org/D16457#715443): this is not a use-case >>> for >>> subpackages. If the main program/library needs to be compiled >>> differently with or without NLS, this is not viable for subpackages. >>> If a port needs to be built multiple times to create different >>> subpackages, this is not a viable case for subpackages. >>> A good candidate is qt6-tools: this port contains multiple tools >>> (designer, linguist, help, and so on). Those tools could be put in >>> different subpackages. >>> >>> I hope this explanation helps to clarify and address some of your >>> concerns. >> >> As I read this, lang/php is the best example of a port where >> subpackages >> has a benefit (in the sense it matches your description above to the >> letter, the main port independent from the subpacakges, and what can >> be >> build as subpackages is a plugin/extension). >> >>> OPTIONS and SUBPACKAGES >>> Do we have to convert OPTIONS to SUBPACKAGES? No. >>> Can a SUBPACKAGE be built only if an OPTION is enabled? Yes. >>> The only viable use cases for SUBPACKAGES being enabled/disabled by >>> OPTIONS is limited to those portions of the port that do not affect >>> the >>> main package. >>> Examples are: additional libraries, additional data files, and so on. >>> >>> Consolidate master/slave ports in one bigger ports >>> About this topic, I guess your concerns are mainly related to >>> potential >>> explosion of build time of the consolidated ports. >>> This is a justified concern. >>> In those cases, we are suggesting to convert slave ports in >>> SUBPACKAGES >>> enabled via OPTIONS. >>> This will allow port builders to configure the bigger ports to not >>> build all SUBPACKAGES, but only the needed ones. This should restore >>> the previous build time. >>> >>> However, as for the php case, the maintainer is going to evaluate if >>> the consolidation makes sense or not. >>> If a consolidation is going to result more problematic than >>> beneficial, >>> it can be reverted and subpackages not adopted for the use case. >> >> If I understand the sum of all the above correct, you suggest to >> remove >> slave ports and to use subpackages instead (where this makes sense in >> terms of the current implementation of subpackages). Or worded >> differently to the same effect (as I only care about the effect and >> not >> the intention), when someone converts a port to subpackages, the >> corresponding slave packages shall be removed (or for new ports: slave >> ports shall not be introduced in this case). >> >> Removing slave ports means we can not depend upon specific parts >> anymore >> when installing from ports, as the subpackages can not be targeted for >> install directly and my example of a subpackages aware php results in >> security implications of to much being installed and active if >> installed >> via make install in /usr/ports/something/with_webinterface. -> best >> example of lang/php to use subpacakges is the best example of why not >> to >> use subpackages / shows what is a regression in the features we >> provide >> in our ports collection. >> >> While qt6-tools may be a good example where subpackages makes sense, >> we >> can not depend on subpacakges for "make install", and as if the port >> would be converted to have subpackages but no slave ports are >> introduced, pkg install and make install would differ in the amount of >> installed files. >> >>> For port builders >>> >>> Port builders can experience longer build times in the future, as >>> master/slave ports could be consolidated in one single larger ports. >>> If this is the case, the larger ports should provide OPTIONS to not >>> build unneeded subpackages. >>> If no OPTION is available, please work with the maintainer to >>> introduce >>> one. >> >> I fail to see the benefit: >> We either lose the possibility to target parts of a package (when >> slave >> ports are removed / not introduced) on make install (with a similar >> amount of build time for the ports tree as it is right now), or get >> higher build times for the package builders. In both cases we do not >> gain something significant. >> >> If we want to keep the (useful in some cases) feature to install from >> ports (there is the case of py39-rpds-py failing to build in my >> poudriere which I tried to debug with the author of py-maturin due to >> a >> runtime issue in maturin, which shows up in my the cross build on >> amd64-intel for amd64-athlon64... which in the end leads me to build >> py-rpds-py on the target machine from ports), the current >> implementation >> of subpackages has to come with the requirement to create slave ports >> for each subpackage. >> >> That's my concern, and that's the reason why I have the opinion, that >> portmgr has to keep the lock on subpackages and reject any subpackage >> which don't come with a slave port. >> >> I would be OK to lift this restriction, when the current >> implementation >> is changed to be able to only install the files of a subpacakge on >> make >> install (an implementation could be to require "make install >> TARGET_SUBPACAKGES=sub1,sub2,sub3" or "make install-subpackage1 >> install-subpacakge2"... as long as recursive dependencies are handled >> according to this requirement, those people designing, implementing >> and >> reviewing this can argue about such details). >> >> Keeping the current implementation (with the restriction of always >> introducing slave packages for subpackages) would need a way to denote >> that a slave port covers a specific subpackage which would allow >> package >> builders to skip those slave ports (and the subpacakges would need to >> have the same package name as the slave port, no idea if this has a >> technical disadvantage in terms of having 2 different origins for the >> same package name, but it surely would be confusing for people at >> first >> look). >> >> TLDR: for the use cases you specified in the beginning, I do not see a >> benefit, only drawbacks. Can you please provide an example of a >> benefit >> I fail to see (yes, more modularity for qt6-tools may be beneficial >> for >> some people, but the cost/benefit between qt6-tools (something which >> we >> don't provide right now) and "make install" (what we provide right >> now) >> or "build time reduction for package builders" (which would have a >> benefit for a lot of use cases) is in my books much more in the >> direction of "make install" and "build time reduction" than towards >> the >> modularization of qt6-tools)? >> >> Bye, >> Alexander. >> >> -- >> http://www.Leidinger.net Alexander@Leidinger.net: PGP >> 0x8F31830F9F2772BF >> http://www.FreeBSD.org netchild@FreeBSD.org : PGP >> 0x8F31830F9F2772BF -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF