git: e497a16a2869 - main - ports-mgmt/pkg*: Release 1.17.0
Bryan Drewery
bdrewery at FreeBSD.org
Sat Jul 24 22:24:29 UTC 2021
We have a storm brewing on the package builders right now. They will
soon be serving repositories with duplicated/multiple versions of packages.
Frankly your response only angered me as it was dismissive and focus
stealing top-post. The details I gave are backed up by many reports
coming in on IRC and github. If you look into your packagesite.yaml and
inspect for both .txz and .pkg you may find duplicates.
I have a life and work priorities of my own. I can't drop everything to
fix this right now. I am skeptical of any fix that does not meticulously
handle the transition and simply tosses in an unlink. Meaning I will
want to see or do my own testing. If I had known the default PKG_EXT was
changing I would have thrown up my arms and blocked this and asked for a
migration period. Poudriere isn't even all there is.
I think users would appreciate having a tool that does the right thing
when the change comes in rather than being forced to go through the
wrong broken thing to get the fixed tool. The longer this is in the
bigger the damage is.
On 7/24/2021 2:50 PM, Kevin Bowling wrote:
> It’s all working fine for me on main and 13.0. Which of these issues
> cannot be solved in the next few days?
>
> On Sat, Jul 24, 2021 at 1:36 PM Bryan Drewery <bdrewery at freebsd.org
> <mailto:bdrewery at freebsd.org>> wrote:
>
> On 7/22/2021 1:06 AM, Baptiste Daroussin wrote:
> > The branch main has been updated by bapt:
> >
> > URL:
> https://cgit.FreeBSD.org/ports/commit/?id=e497a16a286972bfcab908209b11ee6a13d99dc9
> <https://cgit.FreeBSD.org/ports/commit/?id=e497a16a286972bfcab908209b11ee6a13d99dc9>
> >
> > commit e497a16a286972bfcab908209b11ee6a13d99dc9
> > Author: Baptiste Daroussin <bapt at FreeBSD.org>
> > AuthorDate: 2021-07-22 07:54:38 +0000
> > Commit: Baptiste Daroussin <bapt at FreeBSD.org>
> > CommitDate: 2021-07-22 08:05:59 +0000
> >
> > ports-mgmt/pkg*: Release 1.17.0
> >
> > TL;DR: new uniq package file extension: '.pkg'
> >
> > Full changelog for pkg:
> > - add a new "snap" prefix, to deal with pkg base snapshots
> > - pkg repo now accepts packages with different compression formats
> > - pkg now have a single extension: ".pkg" and a backward
> compatible
> > symlink is created when the package is created to help
> transitioning
> > - Default compression level for zstd is now set to 19
> > - the default compression level is now a configuration option
> > - plenty of portability fixes
> > - fix plenty of typos
> > - expose the name of the package to lua and shell scripts
> > - plist:
> > remove internal support for @*exec (this is now in the ports
> tree)
> > remove support for @ignore
> > remove support for @dirrm/@dirrmtry
> > remove support for @pkgdep
> > remove stub support for @stopdaemon
> > remove stub support for @display
> > remove stub support for @mtree
> > remove stub support for @conflict
> > - VUXML: fetch .xz compressed version of the file by default
> > - triggers are deferred to later command (firstboot is
> planned) if run
> > with pkg -r
> > - pkg triggers commands has been added and can execute the
> deferred
> > triggers for example at firstboot
> >
> > Changes in the framework:
> > - Add a backward compatibility layer, so people are not forced
> to move
> > to pkg 1.17 yet (new quarter we will enforce the switch for
> triggers
> > anyway, but it gives time to people to switch)
> > - Introduce a new PKG_COMPRESSION_FORMAT for users to specify the
> > compression format they are willing to use when creating
> packages:
> > txz, tar, tgz, tbz, tzst are the valid ones (note that tzst
> is only
> > usable on FreeBSD 13 and 14)
> > - Add a backward compatibility for people who specified the
> compression
> > format already via PKG_SUFX and issue a warning to tell them
> about the
> > deprecation of PKG_SUFX
> > ---
> > Mk/bsd.port.mk <http://bsd.port.mk> | 58
> +++++++++++++++++++++++--------------------
> > ports-mgmt/pkg-devel/Makefile | 4 +--
> > ports-mgmt/pkg-devel/distinfo | 6 ++---
> > ports-mgmt/pkg/Makefile | 2 +-
> > ports-mgmt/pkg/distinfo | 6 ++---
> > 5 files changed, 40 insertions(+), 36 deletions(-)
> >
> > diff --git a/Mk/bsd.port.mk <http://bsd.port.mk> b/Mk/bsd.port.mk
> <http://bsd.port.mk>
> > index 228644a1845f..5eba5d89da95 100644
> > --- a/Mk/bsd.port.mk <http://bsd.port.mk>
> > +++ b/Mk/bsd.port.mk <http://bsd.port.mk>
> > @@ -1211,6 +1211,14 @@ _OSVERSION_MAJOR=
> ${OSVERSION:C/([0-9]?[0-9])([0-9][0-9])[0-9]{3}/\1/}
> > .if !defined(_PKG_VERSION)
> > _PKG_VERSION!= ${PKG_BIN} -v
> > .endif
> > +# XXX hack for smooth transition towards pkg 1.17
> > +_PKG_BEFORE_PKGEXT!= ${PKG_BIN} version -t
> ${_PKG_VERSION:C/-.*//g} 1.17.0
> > +.if ${_PKG_BEFORE_PKGEXT} == "<"
> > +_PKG_TRANSITIONING_TO_NEW_EXT= yes
> > +_EXPORTED_VARS+= _PKG_TRANSITIONING_TO_NEW_EXT
> > +.warning "It is strongly recommanded to upgrade to a newer
> vertion of pkg first"
> > +.endif
> > +# XXX End of hack
> > _PKG_STATUS!= ${PKG_BIN} version -t
> ${_PKG_VERSION:C/-.*//g} ${MINIMAL_PKG_VERSION}
> > .if ${_PKG_STATUS} == "<"
> > IGNORE= pkg(8) must be version
> ${MINIMAL_PKG_VERSION} or greater, but you have ${_PKG_VERSION}. You
> must upgrade the ${PKG_ORIGIN} port first
> > @@ -2227,23 +2235,29 @@ _PKGMESSAGES+= ${PKGMESSAGE}
> >
> > TMPPLIST?= ${WRKDIR}/.PLIST.mktmp
> >
> > -.if ${WITH_PKG} == devel
> > -PKG_SUFX?= .pkg
> > +# backward compatibility for users
> > +.if defined(_PKG_TRANSITIONING_TO_NEW_EXT)
> > .if defined(PKG_NOCOMPRESS)
> > -PKG_OLDSUFX?= .tar
> > +PKG_SUFX?= .tar
> > .else
> > -#.if ${OSVERSION} > 1400000
> > -#PKG_OLDSUFX?= .tzst
> > -#.else
> > -PKG_OLDSUFX?= .txz
> > -#.endif
> > +PKG_SUFX?= .txz
> > .endif
> > +PKG_COMPRESSION_FORMAT?= ${PKG_SUFX:S/.//}
> > .else
> > +.if defined(PKG_SUFX)
> > +PKG_COMPRESSION_FORMAT?= ${PKG_SUFX:S/.//}
> > +.warning PKG_SUFX is defined, if should be replace with
> PKG_COMPRESSION_FORMAT
> > +.endif
> > +PKG_SUFX= .pkg
> > +.endif
>
> I love the idea here but can this be reverted for now please? Poudriere
> is not prepared for a default PKG_EXT change.
>
> 1. Now that it finds a new PKG_EXT of .pkg, it cannot find the existing
> bootstrap pkg.txz because it looks for Latest/pkg.pkg ->
> ../All/pkg-*.pkg even though the *existing* one is .txz. I'm not sure
> how this symlink is added before pkg is even built but it's not done in
> a compatible way.
>
> 2. The new PKG_EXT makes Poudriere now find no existing packages so
> everything is rebuilt. Not a deal breaker in itself but a problem for
> future builds...
>
> 3. The rebuild won't remove all the .txz files. I assume once `pkg repo`
> runs it will find both .txz and .pkg files and cause duplicate version
> problems in `pkg repo`
>
> I see something creates a .txz -> .pkg symlink. What deles that symlink
> if Poudriere only deletes/rebuilds the .pkg?
>
> ***
> "all those .txz entries are actually _duplicates_ in packagesite.yaml"
> seen on IRC just now by someone.
> I can confirm that looking in a packagesite.yaml I just generated. Both
> the older .txz version of a port and a new revision-bumped one appear in
> the yaml.
> ***
>
> 4. I've seen reports of packagesite.pkg which is concerning because
> Poudriere manages packagesite.txz with its own symlinks for
> hardlink-copied repositories. It does not use PKG_EXT there as they
> weren't packages, but I suspect we need special handling even with
> PKG_EXT added there.
>
> --
> Regards,
> Bryan Drewery
>
--
Regards,
Bryan Drewery
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/dev-commits-ports-all/attachments/20210724/0ded79d7/attachment.sig>
More information about the dev-commits-ports-all
mailing list