From nobody Wed Jan 24 09:28:48 2024 X-Original-To: freebsd-ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TKdv35rdkz57tBs; Wed, 24 Jan 2024 09:29:03 +0000 (UTC) (envelope-from luca.pizzamiglio@gmail.com) Received: from mail-io1-f45.google.com (mail-io1-f45.google.com [209.85.166.45]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TKdv25gXWz41g9; Wed, 24 Jan 2024 09:29:02 +0000 (UTC) (envelope-from luca.pizzamiglio@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=freebsd.org (policy=none); spf=pass (mx1.freebsd.org: domain of luca.pizzamiglio@gmail.com designates 209.85.166.45 as permitted sender) smtp.mailfrom=luca.pizzamiglio@gmail.com Received: by mail-io1-f45.google.com with SMTP id ca18e2360f4ac-7bed9f5d35dso258294939f.3; Wed, 24 Jan 2024 01:29:02 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706088541; x=1706693341; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=9zuqbbAM8w11+ds45CiaFSx3hAkqp5W0exGNA5DV0DQ=; b=NSJE4X9mTKGSU4iJDUzbza1iMGpjgWhATgv09BYxjnzl+aCBtHcrH7ORgdzEvLBIx1 2vACCgYLopk40ssBfyGV/rf5n45s2W0fmCawoO8P9a2IOB8qIP9oJ7CwDkIuCljMvoxd 1tYl0oFOb/jv1hy5tyUwS3jjsq/tYgAK1JRAIKRoUmx/3iqz2kk4uQg6bHRU70XjfAlA 4JtOdhlW0MsqLC4/mK/AQbC//0JevnDP+ev2A0DfKzK6V13JKwEtt/YmFHG5nAmvVWyi qwiwoRPE04p9wbC5Du0iczumWph5w82zowpwi8GMhF+/oc6FOhxkn7Zlv91t2avN+KDe sNqQ== X-Gm-Message-State: AOJu0YynAbH8/VnHXLDMIxYj8WV2BW0k14F1kPhY3VvHvsjDtljEdZ4d 7ZvmgSRkZ0u0UXT0qxcxSyutxrzjR1FlU/efNMbT/e353q/yh72dqeYIe37MHfs= X-Google-Smtp-Source: AGHT+IFSSXkQN7yadpSEAkE+KXKo8zo6bd5uwYtB+AIYkG03tzNRX30UVrsFZFcHX6VlULa9YRhJew== X-Received: by 2002:a92:c889:0:b0:362:9066:c77e with SMTP id w9-20020a92c889000000b003629066c77emr1182488ilo.61.1706088540628; Wed, 24 Jan 2024 01:29:00 -0800 (PST) Received: from mail-io1-f46.google.com (mail-io1-f46.google.com. [209.85.166.46]) by smtp.gmail.com with ESMTPSA id z13-20020a92cd0d000000b0036192c17459sm5288961iln.3.2024.01.24.01.29.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 24 Jan 2024 01:29:00 -0800 (PST) Received: by mail-io1-f46.google.com with SMTP id ca18e2360f4ac-7bed9f5d35dso258294639f.3; Wed, 24 Jan 2024 01:29:00 -0800 (PST) X-Received: by 2002:a92:da8f:0:b0:360:8d3f:9a45 with SMTP id u15-20020a92da8f000000b003608d3f9a45mr1222306iln.116.1706088540045; Wed, 24 Jan 2024 01:29:00 -0800 (PST) List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org MIME-Version: 1.0 From: Luca Pizzamiglio Date: Wed, 24 Jan 2024 10:28:48 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Subpackage explanations To: FreeBSD Ports mailing list , freebsd-ports Content-Type: multipart/alternative; boundary="000000000000628b99060fadb173" X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.90 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[pizzamig@freebsd.org,lucapizzamiglio@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : SPF not aligned (relaxed), No valid DKIM,none]; MIME_TRACE(0.00)[0:+,1:+,2:~]; MISSING_XM_UA(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.166.45:from]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; ARC_NA(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; R_DKIM_NA(0.00)[]; TAGGED_FROM(0.00)[]; FROM_NEQ_ENVFROM(0.00)[pizzamig@freebsd.org,lucapizzamiglio@gmail.com]; FROM_HAS_DN(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; TO_DN_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-ports@freebsd.org,ports@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; RCVD_IN_DNSWL_NONE(0.00)[209.85.166.45:from,209.85.166.46:received] X-Rspamd-Queue-Id: 4TKdv25gXWz41g9 --000000000000628b99060fadb173 Content-Type: text/plain; charset="UTF-8" Hi porters! At the beginning of January, we merged the support to subpackages in the framework. Subpackage is the feature to create multiple packages from one build of one port. In other words, now it's possible to group files into multiple packages. This means that from one port it's possible to split the build into several packages. Some additional details are available in this lighting talk at EuroBSD 2023 (https://youtu.be/e-FUYbGNdBg?t=824). *Use cases we want to tackle* The first use case we want to get rid of is master/slave ports when slave ports could be built with the master port. An example (already merged) is https://reviews.freebsd.org/D43445: devel/appstream will create also the -compose and -qt6 binaries as subpackage in one build. Another use case is ports that could be split in smaller units. An example is devel/qt6-tools: this port contains all qt6 related utilities. The package could be split into smaller subpackages, one per tool. Ports that depend on one of the qt6 tools could then install only the smaller subpackage, instead of the whole collection of tools. Those first use cases should provide some benefit to build times (i.e. extract/patch/configure will happen only once, build dependencies are smaller, then faster to install) on the package clusters, but they could increase build times for folks building ports. *Use cases we don't want to tackle (yet)* Subpackages enable the adoption of micro-subpackages, a typical pattern for Linux distributions that split a package in smaller ones: one with docs (-doc), one with static libraries and headers (-dev), one with manpages (-man), one with examples (-examples), and so on. While technically this could be easy to do, it's a major change for users and there needs to be a discussion in the community on whether it is a desirable feature. For this reason, we are not proceeding with this *SUBPACKAGES and OPTIONS* Subpackages can affect build time negatively for users that don't use default packages, but they build packages directly from ports. Let's take the appstream port as an example. Before subpackages, you could selectively build appstream and anything else. After subpackages, the appstream port is going to build appstream, appstream-compose and appstream-qt6 (and the related dependencies). This can considerably increase build time for some folks. One way to mitigate it is to add a mechanism to make subpackages optional, via OPTIONS (currently supported). An OPTION for -compose and one for -qt6 can enable/disable those subpackages, restoring the shorter build time. However, optional subpackages can cause unexpected results when other ports depend on subpackages. Let's imagine a port depending on appstream-compose, but appstream has the compose OPTION disabled. There are two possible approaches: 1. the build of the port is going to fail as appstream is not building the dependency (the subpackage is disabled by the OPTION) 2. the build is going to change the configuration of appstream, enabling the compose OPTION, to build the dependency. We chose the approach #1: even if the build fails, it shows which package fails to be installed, providing insights on how to fix it. Approach #2 introduces (hidden) side effects and surprises, like unexpected dependencies or custom configuration ignored. The benefit is that the dependent port is successfully built, ignoring the effect of changing configuration of the dependencies. As both approaches have their own pros and cons, anyone can have a different opinion on that. To limit the effects on the selected approach: * all options enabling subpkg has to be ON by default, to not introduce broken dependencies * we encourage to limit the introduction of those optional subpackages, limiting their adoption only for cases where the related subpackage adds a relevant set of dependencies Additionally, the approach #1 could be improved by detecting such use cases and providing a meaningful error message. *Adoption and documentation* Subpackages are already in the framework, but while we start to adopt them for a couple of ports, we are still finding a few details that needs to be clarified (i.e. MOVED). Subpackages' adoption is blocked atm by a git hook (server side). The hook can be unblocked via portmgr approval (push option allow-subpkg). We encourage port committers to start thinking which of their ports is suitable to use subpackages or not. We don't have official documentation yet, I'm going to work on that in the next few weeks. On phabricator, there is a patch (https://reviews.freebsd.org/D43079) containing two fake ports, used to build and test subpackages. They provide the best example for now of how subpackages look like. *Tooling support* poudriere already supports subpackages. portupgrade, portmaster and synth don't support subpackages. We kindly ask the maintainers of those projects to add the support to subpackages. Support in portlint is in progress. A git commit hook will be developed to ensure that all subpackages are built by default. Best regards, pizzamig on behalf of portmgr --000000000000628b99060fadb173 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi porters!

At the beg= inning of January, we merged the support to subpackages in the framework.
Subpackage is the feature to create multiple packages from one build of one port. In = other=20 words, now it's possible to group files into multiple packages. This=20 means that from one port it's possible to split the build into several = packages.
Some additional details are available in this lighting = talk at EuroBSD 2023 (https://youtu.be/e-FUYbGNdBg?t=3D824).

Use cases we want to tackle
The first use case we w= ant to get rid of is master/slave ports when slave ports could be built wit= h the master port.
An example (already merged) is=C2=A0https://reviews.free= bsd.org/D43445: devel/appstream will create also the -compose and -qt6 = binaries as subpackage in one build.

Another use case is ports that could be split in smaller units. An example is=20 devel/qt6-tools: this port contains all qt6 related utilities. The=20 package could be split into smaller subpackages, one per tool.
Po= rts that depend on one of the qt6 tools could then install only the=20 smaller subpackage, instead of the whole collection of tools.
Those first use cases should provide some benefit to build times (i.e.=20 extract/patch/configure will happen only once, build dependencies are=20 smaller, then faster to install) on the package clusters, but they could in= crease build times for folks building ports.

<= b>Use cases we don't want to tackle (yet)
Subpackages enable the adoption of micro-subpackages, a typical pattern for Linux=20 distributions that split a package in smaller ones: one with docs=20 (-doc), one with static libraries and headers (-dev), one with manpages=20 (-man), one with examples (-examples), and so on.
While=20 technically this could be easy to do, it's a major change for users and= =20 there needs to be a discussion in the community on whether it is a desirabl= e feature. For this reason, we are not proceeding with this

<= div>SUBPACKAGES and OPTIONS
Subpackages can affect build t= ime negatively for users that don't use default packages, but they buil= d packages directly from ports.
Let's take the appstream = port as an example.
Before subpackages, you could selectively bui= ld appstream and anything else.
After subpackages, the appstream port is going to build appstream,=20 appstream-compose and appstream-qt6 (and the related dependencies).
This can considerably increase build time for some folks.
One way to mitigate it is to add a mechanism to make subpackages optional,=20 via OPTIONS (currently supported). An OPTION for -compose and one for=20 -qt6 can enable/disable those subpackages, restoring the shorter build time= .
However, optional subpackages can cause unexpected results when= other ports depend on subpackages.
Let's imagine a port depe= nding on appstream-compose, but appstream has the compose OPTION disabled.<= br>
There are two possible approaches:
1. the build of = the port is going to fail as appstream is not building the dependency (the = subpackage is disabled by the OPTION)
2. the build is going t= o change the configuration of appstream, enabling the compose OPTION, to bu= ild the dependency.

We chose the approach #1: even if the build fails, it shows which package=20 fails to be installed, providing insights on how to fix it.
A= pproach #2 introduces (hidden) side effects and surprises, like unexpected=20 dependencies or custom configuration ignored. The benefit is that=20 the dependent port is successfully built, ignoring the effect of changing c= onfiguration of the=20 dependencies.
As both approaches have their own pros and cons= , anyone can have a different opinion on that.
To limit the effects on t= he selected approach:
* all options enabling subpkg has to be ON by defa= ult, to not introduce broken dependencies
* we encourage to limit the introduction of those optional subpackages, limi= ting their adoption only for cases where the related subpackage adds a relevant=20 set of dependencies
Additionally, the approach #1 could be im= proved by detecting such use cases and providing a meaningful error message= .

Adoption and documentation
Subpac= kages are already in the framework, but while we start to adopt them for a=20 couple of ports, we are still finding a few details that needs to be=20 clarified (i.e. MOVED).
Subpackages' adoption is blocked= =20 atm by a git hook (server side). The hook can be unblocked via portmgr=20 approval (push option allow-subpkg).
We encourage port committers= to start thinking which of their ports is suitable to use subpackages or n= ot.
We don't have official documentation yet, I'm goi= ng to work on that in the next few weeks.
On phabricator, there is= a patch (= https://reviews.freebsd.org/D43079) containing two fake ports, used to build and test subpackages.
T= hey=20 provide the best example for now of how subpackages look like.
Tooling support
poudriere already support= s subpackages.
portupgrade, portmaster and synth don't support subpackages. We kindly ask the=20 maintainers of those projects to add the support to subpackages.
= Support in portlint is in progress.
A git commit hook will be dev= eloped to ensure that all subpackages are built by default.
<= div>
Best regards,
pizzamig on behalf of p= ortmgr
--000000000000628b99060fadb173--