From nobody Sun Dec 29 17:56:41 2024 X-Original-To: dev-commits-doc-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YLn3t12JYz5jWgZ for ; Sun, 29 Dec 2024 17:56:42 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YLn3s45r6z46kk; Sun, 29 Dec 2024 17:56:41 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1735495001; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=vy3NZkJdTmaclGICtLThpJR2qjDP75NjlHALKEqjuog=; b=YqSke3yjJKKOf/Gsx1sZfkUx/0p47k2ZeWm0DOkDEluTnmwrFV31022lBC4xO4ahoRKBQ9 NcdaKPDj1qNmCCC+OR7lzSBiyk8p/AmR6eS9Jp5ztaeWD+SGlQ1cuTdrF7WYpLGToLB0vq d3fxsBfb2+vuskite/knU29Ujt/HKhBwYel5lKGGlLfgxkG8HI2PjRMdAzvtZ8pcByLo2a 2ewj4i8ZTpj4ienWBcoGLEvGHdOoGsChxkRI3z0O0SuyrTdAp4cUyBI8sRMYAcpCRtltgu wDg8IUlYIJaIJdXH0er6bp+IJzbT3ICWZZ0+aizgem2TsUv8ES12UA4Y2wejOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1735495001; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=vy3NZkJdTmaclGICtLThpJR2qjDP75NjlHALKEqjuog=; b=HbqIKHqKRuKaeIYsPi3N/aQ5sNW0NTZvpsBVSYsVA8lyA68q/1ssXyp1fCgz1DvQvtsTm2 +OauctxkL0Gr1EiJWWWxQ+hEOWnM2ztajIOMRK/FbenyWHkj2jW9QNONrZ4LVArsw1OMnv /ngr3iPY2/JpnAuaPI0EnF2uk+JmMSXh9l3Vd6hCDcc/2UcCrBL2e9iCv20Jf7tHnPdb21 zU4qkbBsCVOsaZ9j4DaAeA4qk6LLX9QAqewZx1dGEgKcpp3uQY9nh7oeObLOK78VQ6zUz8 JwxqIjvdx5el8p1a29c0xA/3SxrtTcgmANMifv0yO20zT9WLqMNcLWBxu62pow== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1735495001; a=rsa-sha256; cv=none; b=XO7lQXGEsCgLnlIhI6qimbfDwel3JjEpkjhGfinAmQlUcvVNmZY17GcN8+ZPqJu/tFE2MV 9yYwYm0azi4hK2lCcs74ikEqiJdxb4RnxqLBQBYNHc+3n60RNU+8oocPwTUx5riYzB9Mgb zwKVTo1Kf47suA1EMBVtnS9vnhkOHyUuDFya/7UdfzeOlRIEoWmbAaTXZwDntiwAnvtwec 860TzfO8JJRLt3jqG+IVljM5bI/Ltk0FoG9JMAIaeTkYgGOtjiXW8qiEcgA7FVVt4Vq+Qb Gv3MPDmotkbwic52s4JSZi1w5mlnz2DZyxEghQKzMXOyPjftX+OjKaTycnXZXA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4YLn3s3ZGGzJfS; Sun, 29 Dec 2024 17:56:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 4BTHufos017252; Sun, 29 Dec 2024 17:56:41 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 4BTHuf0I017249; Sun, 29 Dec 2024 17:56:41 GMT (envelope-from git) Date: Sun, 29 Dec 2024 17:56:41 GMT Message-Id: <202412291756.4BTHuf0I017249@gitrepo.freebsd.org> To: doc-committers@FreeBSD.org, dev-commits-doc-all@FreeBSD.org From: Sergio Carlavilla Delgado Subject: git: dfc8a64db2 - main - Status Reports: Add pkgbase report List-Id: Commit messages for all branches of the doc repository List-Archive: https://lists.freebsd.org/archives/dev-commits-doc-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-doc-all@freebsd.org Sender: owner-dev-commits-doc-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: carlavilla X-Git-Repository: doc X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: dfc8a64db28317bcf499003ba940f9af4820e010 Auto-Submitted: auto-generated The branch main has been updated by carlavilla: URL: https://cgit.FreeBSD.org/doc/commit/?id=dfc8a64db28317bcf499003ba940f9af4820e010 commit dfc8a64db28317bcf499003ba940f9af4820e010 Author: Sergio Carlavilla Delgado AuthorDate: 2024-12-29 17:55:12 +0000 Commit: Sergio Carlavilla Delgado CommitDate: 2024-12-29 17:55:12 +0000 Status Reports: Add pkgbase report Reviewed by: Pau Amma Pull Request: https://github.com/freebsd/freebsd-doc/pull/444 --- .../status/report-2024-10-2024-12/pkg-pkgbase.adoc | 34 ++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/website/content/en/status/report-2024-10-2024-12/pkg-pkgbase.adoc b/website/content/en/status/report-2024-10-2024-12/pkg-pkgbase.adoc new file mode 100644 index 0000000000..86b3c84265 --- /dev/null +++ b/website/content/en/status/report-2024-10-2024-12/pkg-pkgbase.adoc @@ -0,0 +1,34 @@ +=== PkgBase-motivated improvements to pkg + +Contact: Isaac Freund + +Some problems blocking progress on the link:https://wiki.freebsd.org/PkgBase[PkgBase project] are caused by shortcomings of man:pkg[8]. +The primary goal of my work on pkg is to unblock PkgBase progress. +However, all users of pkg will benefit even if they do not use PkgBase. + +The scheduler for pkg's install/upgrade/delete jobs has been link:https://github.com/freebsd/pkg/pull/2330[rewritten], motivated by solving link:https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=259785[PR259785]. +The new scheduler models the scheduling problem as a directed graph and splits upgrade jobs into delete/install halves only when necessary to break a cycle in the graph. +This formal model gives strong guarantees about ordering that the old scheduler was not able to provide and prevents unnecessary splitting of upgrade jobs. +It also fixes longstanding bugs where the old scheduler would bail out and cause the entire upgrade to fail. +The new scheduler is included in pkg version 1.21.99.3 (pkg-devel). + +The rest of my work this quarter has been related to pkg's automatic tracking of shared library dependencies, which PkgBase heavily relies on. +The initial motivating problem was link:https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265061[PR265061] but it was necessary to make more fundamental changes to how pkg tracks shlibs before cleanly solving that problem became possible. + +When a package is created with man:pkg-create[8], pkg scans the included files and generates shlibs_provided/shlibs_required lists based on the executables/shared libraries found. +Before my changes, pkg would use the elf hints file of the host system as an input to pkg-create in order to filter out shlibs provided by the base system from the generated shlibs_required list. +An ALLOW_BASE_SHLIBS option disabled this filtering for the purpose of building PkgBase packages. + +After my changes, pkg-create no longer reads the elf hints file of the host system and base system shlibs are included in the generated shlibs_required list. +When man:pkg-install[8]/man:pkg-upgrade[8]/etc. invoke the solver on an non-PkgBase system, pkg generates a list of shlibs provided by the base system as an input to the solver by scanning /lib and /usr/lib. +On a PkgBase system, the PkgBase packages provide all base system shlibs. + +This allows the ALLOW_BASE_SHLIBS option to be eliminated. +It also gives better integration between the ports packages and PkgBase packages as shlib dependencies of ports packages on PkgBase packages are now tracked rather than ignored. +Finally, this change significantly simplifies the pkg codebase and improves portability. +This change was implemented in link:https://github.com/freebsd/pkg/pull/2386[] and is not yet included in a pkg release. + +With that change and other link:https://github.com/freebsd/pkg/pull/2376[internal improvements] I was able to add support for tracking lib32 and Linuxulator shlibs, which should resolve the problem that originally motivated my work on pkg's shlib handling (link:https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265061[PR265061]). +This support is implemented in link:https://github.com/freebsd/pkg/pull/2387[] and is not yet included in a pkg release. + +Sponsor: The FreeBSD Foundation