From nobody Mon Jun 05 20:18:40 2023 X-Original-To: questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QZlLK0Cxxz4b5x8 for ; Mon, 5 Jun 2023 20:18:49 +0000 (UTC) (envelope-from sysadmin.lists@mailfence.com) Received: from wilbur.contactoffice.com (wilbur.contactoffice.com [212.3.242.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4QZlLH4t6Mz40F2 for ; Mon, 5 Jun 2023 20:18:47 +0000 (UTC) (envelope-from sysadmin.lists@mailfence.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=mailfence.com header.s=20210208-e7xh header.b="O/PTD5LV"; spf=pass (mx1.freebsd.org: domain of sysadmin.lists@mailfence.com designates 212.3.242.68 as permitted sender) smtp.mailfrom=sysadmin.lists@mailfence.com; dmarc=pass (policy=quarantine) header.from=mailfence.com Received: from ichabod.co-bxl (ichabod.co-bxl [10.2.0.36]) by wilbur.contactoffice.com (Postfix) with ESMTP id EDF039F9 for ; Mon, 5 Jun 2023 22:18:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1685996323; s=20210208-e7xh; d=mailfence.com; i=sysadmin.lists@mailfence.com; h=Date:From:To:Message-ID:In-Reply-To:References:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding; l=5659; bh=86X4LqV8pDjGkYMkMRfLICGxKneSTbSVxVPW+nv/dTs=; b=O/PTD5LVpP1xa/4Vn6bbPfC+B8sHJDa79J2Tv2hVKQQ/iQ86l6MUYXYFrOHLSElU 6dphEJ+5nTxrV0wEqXUcl/F82IuaOtKtNFwmTYoWb+kwV9w9umnugrVe7pj3yPKf/om RMHhcZYrEfCaFMHRNo30dT6F3DoCELL+SSdFISTH9qdBLxVToYpNyzVAf0iQhpOp0lC wYn3N09tod3R2dOg2tRRA1g1wbXRUZRE6AUSeKv6knvtyvw+tB9rsBY32eFuGcBGb41 3ABUG6mj4aH61hBbPsOCI/fTRZtP8L3q2N9A+UPuNovArCVfpw02meMqQCGkpbnYDh+ VZlNWdECVw== Date: Mon, 5 Jun 2023 22:18:40 +0200 (CEST) From: Sysadmin Lists To: questions@freebsd.org Message-ID: <1368945115.467084.1685996320460@ichabod.co-bxl> In-Reply-To: <1653882830.656908.1685438569298@ichabod.co-bxl> References: <1653882830.656908.1685438569298@ichabod.co-bxl> Subject: Re: pkg-ugprade checksums List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Mailer: ContactOffice Mail X-ContactOffice-Account: com:312482426 X-Spamd-Result: default: False [-4.08 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.994]; DMARC_POLICY_ALLOW(-0.50)[mailfence.com,quarantine]; R_SPF_ALLOW(-0.20)[+ip4:212.3.242.64/26]; R_DKIM_ALLOW(-0.20)[mailfence.com:s=20210208-e7xh]; RCVD_IN_DNSWL_LOW(-0.10)[212.3.242.68:from]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; MLMMJ_DEST(0.00)[questions@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:10753, ipnet:212.3.242.64/26, country:US]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[mailfence.com:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[questions@freebsd.org]; ARC_NA(0.00)[] X-Rspamd-Queue-Id: 4QZlLH4t6Mz40F2 X-Spamd-Bar: ---- X-ThisMailContainsUnwantedMimeParts: N Top-posting a partial answer since I'm the OP. Finally had some time to com= pile `pkg' with full debug symbols and run it under gdb. # pkg info vim-x11 | grep Insta Installed on : Mon Apr 17 21:24:26 2023 PDT # stat -f %Sm tmpdir/pkg_project/All/vim-x11-9.0.1366.pkg=20 May 27 02:17:07 2023 Which means the package was installed prior to the newest build. Checking the local repo database: # pkg shell sqlite> .open /var/db/pkg/repo-Poudriere-quarterly.sqlite sqlite> select name,version,cksum from packages where name is 'vim-x11'; name =3D vim-x11 version =3D 9.0.1366 cksum =3D ae560492c77e7629a4c7b8162b256acbd39e4656c55516afcb8729121afc9= 4dd The cksum matches the sha256 fingerprint of the recently built package. Which means: 1. My local repo is up-to-date. 2. My local cache isn't. 3. pkg doesn't notice. pkg_checksum.c:pkg_checksum_generate() describes how the fingerprint is generated for `pkg query %X' (the "internal package checksum"): Thread 2.1 hit Breakpoint 2, pkg_checksum_calculate (pkg=3D0x801519800, db= =3D0x0, inc_scripts=3Dtrue, inc_version=3Dfalse, inc_files=3Dtrue) at /wrkdirs/usr/ports/ports-mgmt/pkg/work/pkg-1.19.1/libpkg/pkg_checksu= m.c:606 (gdb) c Continuing. =20 Thread 2.1 hit Breakpoint 1, pkg_checksum_generate (pkg=3D0x801519800, dest= =3D0x8014416f0 "h", destlen=3D112, type=3DPKG_HASH_TYPE_BLAKE2_BASE32, inc_= scripts=3Dtrue, inc_version=3Dfalse, inc_files=3Dtrue) at /wrkdirs/usr/ports/ports-mgmt/pkg/work/pkg-1.19.1/= libpkg/pkg_checksum.c:189 (gdb) break pkg_checksum.c:553 Breakpoint 3 at 0x4b1b15: pkg_checksum.c:553. (2 locations)=20 (gdb) c =20 Continuing. =20 Thread 2.1 hit Breakpoint 3.1, pkg_checksum_encode_base32 ( in=3D0x801516900 "\246n&_K5\031\322\025R\242\347\tt\265y`\234\251*\303a= \t\\\2558\334`\345e\300&\301o:H\253=C9=9A\3460\220\351\205T\250", inlen=3D6= 4, out=3D0x8014416f4 "gi5c1xpjijgr7kekn73uy4is3daa3wkfdqa1yqiiabzbs1zcysjn= hzj8e4kucp4hobru6n1kefyhj8gobouo1w96xb3fjzqafc9b8f", outlen=3D108) at /wrkdirs/usr/ports/ports-mgmt/pkg/work/pkg-1.19.1/libpkg/pkg_checksu= m.c:553 These are the variables used to generate the checksum: /* =20 * At the moment we use the following fields to calculate the unique checks= um * of the following fields: * - name * - origin * - version * - arch * - options * - required_shlibs * - provided_shlibs * - users * - groups * - dependencies */ =20 For vim-x11, it's "archfreebsd:13:x86:64namevim-x11origineditors/vim" =20 And letting it run to completion, we get: =20 1: pkg->digest =3D 0x801441450 "2$2$bpgamyc8d17g7qm1orycj4qjq18xe9kbzhqctwu= qugejakj7ppqoo7i7kokimc33h5ejb168nb13m7tgr1ggang4krgcsyicskzjs1y" 2: new_digest =3D 0x8014416f0 "2$2$gi5c1xpjijgr7kekn73uy4is3daa3wkfdqa1yqii= abzbs1zcysjnhzj8e4kucp4hobru6n1kefyhj8gobouo1w96xb3fjzqafc9b8fb" pkg->digest is the value found inside local.sqlite (named "manifestdigest")= , and new_digest is the one generated during this run. There's a directive to update the database with the new digest value, but i= t doesn't get executed on '%X' queries. Now, to ask pkg@ what the intention behind that is. > ---------------------------------------- > From: Sysadmin Lists > Date: May 30, 2023, 2:22:49 AM > To: > Subject: pkg-ugprade checksums >=20 >=20 > How does pkg-upgrade check checksums? >=20 > From man 8 pkg-upgrade: >=20 > Packages are fetched from the repositories into the local package ca= che > if they are not already present, or if the checksum of the cached pa= ckage > file differs from the one in the repository. >=20 > But, the packagesite.yaml of my Poudriere repo has: > "name":"vim-x11" > "version":"9.0.1366" > "sum":"ae560492c77e7629a4c7b8162b256acbd39e4656c55516afcb8729121afc94dd" >=20 > And the locally cached package has: > # sha256 -r /var/cache/pkg/vim-x11* > 3bbe31951bc5fd6b08c412fcb0c6f8b494cdb15eafdd4bbacdf1657fe567af43 /var/cac= he/pkg/vim-x11-9.0.1366.pkg >=20 > Yet it doesn't get upgraded: >=20 > # pkg update -fr Poudriere-quarterly > Updating Poudriere-quarterly repository catalogue... > Fetching meta.conf: 100% 163 B 0.2kB/s 00:01 =20 > Fetching packagesite.pkg: 100% 153 KiB 157.1kB/s 00:01 =20 > Processing entries: 100% > Poudriere-quarterly repository update completed. 568 packages processed. > All repositories are up to date. >=20 > # pkg upgrade -r Poudriere-quarterly vim-x11 > Updating Poudriere-quarterly repository catalogue... > Poudriere-quarterly repository is up to date. > All repositories are up to date. > Checking integrity... done (0 conflicting) > Your packages are up to date. >=20 > # truss -fs 256 pkg update -fr Poudriere-quarterly=20 > Shows packagesite.yaml.pkg downloaded to /tmp/, presumably runs its check= s, > then deletes it. >=20 > If I force re-installation, it downloads the new package and installs it. > # pkg install -fr Poudriere-quarterly vim-x11=20 >=20 > So, why is pkg-upgrade's checksums check failing? And why isn't there a > persistent packagesite.pkg on the client for pkg-update to compare to (to= see > if it needs updating) as implied by: >=20 > Package repository catalogues will be automatically updated whenever= pkg > upgrade is run by a user ID with write access to the package databas= e >=20 > I looked at pkghash.h and upgrade.c but don't see where the hash check is > failing. >=20 >=20 > --=20 > Sent with https://mailfence.com =20 > Secure and private email >=20 --=20 Sent with https://mailfence.com =20 Secure and private email