git: e497a16a2869 - main - ports-mgmt/pkg*: Release 1.17.0

[Kevin Bowling]

On Sat, Jul 24, 2021 at 1:36 PM Bryan Drewery <bdrewery at freebsd.org> wrote:
>
> On 7/22/2021 1:06 AM, Baptiste Daroussin wrote:
> > The branch main has been updated by bapt:
> >
> > URL: https://cgit.FreeBSD.org/ports/commit/?id=e497a16a286972bfcab908209b11ee6a13d99dc9
> >
> > commit e497a16a286972bfcab908209b11ee6a13d99dc9
> > Author:     Baptiste Daroussin <bapt at FreeBSD.org>
> > AuthorDate: 2021-07-22 07:54:38 +0000
> > Commit:     Baptiste Daroussin <bapt at FreeBSD.org>
> > CommitDate: 2021-07-22 08:05:59 +0000
> >
> >     ports-mgmt/pkg*: Release 1.17.0
> >
> >     TL;DR: new uniq package file extension: '.pkg'
> >
> >     Full changelog for pkg:
> >     - add a new "snap" prefix, to deal with pkg base snapshots
> >     - pkg repo now accepts packages with different compression formats
> >     - pkg now have a single extension: ".pkg" and a backward compatible
> >       symlink is created when the package is created to help transitioning
> >     - Default compression level for zstd is now set to 19
> >     - the default compression level is now a configuration option
> >     - plenty of portability fixes
> >     - fix plenty of typos
> >     - expose the name of the package to lua and shell scripts
> >     - plist:
> >       remove internal support for @*exec (this is now in the ports tree)
> >       remove support for @ignore
> >       remove support for @dirrm/@dirrmtry
> >       remove support for @pkgdep
> >       remove stub support for @stopdaemon
> >       remove stub support for @display
> >       remove stub support for @mtree
> >       remove stub support for @conflict
> >     - VUXML: fetch .xz compressed version of the file by default
> >     - triggers are deferred to later command (firstboot is planned) if run
> >       with pkg -r
> >     - pkg triggers commands has been added and can execute the deferred
> >       triggers for example at firstboot
> >
> >     Changes in the framework:
> >     - Add a backward compatibility layer, so people are not forced to move
> >       to pkg 1.17 yet (new quarter we will enforce the switch for triggers
> >       anyway, but it gives time to people to switch)
> >     - Introduce a new PKG_COMPRESSION_FORMAT for users to specify the
> >       compression format they are willing to use when creating packages:
> >       txz, tar, tgz, tbz, tzst are the valid ones (note that tzst is only
> >       usable on FreeBSD 13 and 14)
> >     - Add a backward compatibility for people who specified the compression
> >       format already via PKG_SUFX and issue a warning to tell them about the
> >       deprecation of PKG_SUFX
> > ---
> >  Mk/bsd.port.mk                | 58 +++++++++++++++++++++++--------------------
> >  ports-mgmt/pkg-devel/Makefile |  4 +--
> >  ports-mgmt/pkg-devel/distinfo |  6 ++---
> >  ports-mgmt/pkg/Makefile       |  2 +-
> >  ports-mgmt/pkg/distinfo       |  6 ++---
> >  5 files changed, 40 insertions(+), 36 deletions(-)
> >
> > diff --git a/Mk/bsd.port.mk b/Mk/bsd.port.mk
> > index 228644a1845f..5eba5d89da95 100644
> > --- a/Mk/bsd.port.mk
> > +++ b/Mk/bsd.port.mk
> > @@ -1211,6 +1211,14 @@ _OSVERSION_MAJOR=      ${OSVERSION:C/([0-9]?[0-9])([0-9][0-9])[0-9]{3}/\1/}
> >  .if !defined(_PKG_VERSION)
> >  _PKG_VERSION!=       ${PKG_BIN} -v
> >  .endif
> > +# XXX hack for smooth transition towards pkg 1.17
> > +_PKG_BEFORE_PKGEXT!= ${PKG_BIN} version -t ${_PKG_VERSION:C/-.*//g} 1.17.0
> > +.if ${_PKG_BEFORE_PKGEXT} == "<"
> > +_PKG_TRANSITIONING_TO_NEW_EXT=       yes
> > +_EXPORTED_VARS+=     _PKG_TRANSITIONING_TO_NEW_EXT
> > +.warning "It is strongly recommanded to upgrade to a newer vertion of pkg first"
> > +.endif
> > +# XXX End of hack
> >  _PKG_STATUS!=        ${PKG_BIN} version -t ${_PKG_VERSION:C/-.*//g} ${MINIMAL_PKG_VERSION}
> >  .if ${_PKG_STATUS} == "<"
> >  IGNORE=              pkg(8) must be version ${MINIMAL_PKG_VERSION} or greater, but you have ${_PKG_VERSION}. You must upgrade the ${PKG_ORIGIN} port first
> > @@ -2227,23 +2235,29 @@ _PKGMESSAGES+=        ${PKGMESSAGE}
> >
> >  TMPPLIST?=   ${WRKDIR}/.PLIST.mktmp
> >
> > -.if ${WITH_PKG} == devel
> > -PKG_SUFX?=   .pkg
> > +# backward compatibility for users
> > +.if defined(_PKG_TRANSITIONING_TO_NEW_EXT)
> >  .if defined(PKG_NOCOMPRESS)
> > -PKG_OLDSUFX?=        .tar
> > +PKG_SUFX?=   .tar
> >  .else
> > -#.if ${OSVERSION} > 1400000
> > -#PKG_OLDSUFX?=       .tzst
> > -#.else
> > -PKG_OLDSUFX?=        .txz
> > -#.endif
> > +PKG_SUFX?=   .txz
> >  .endif
> > +PKG_COMPRESSION_FORMAT?=     ${PKG_SUFX:S/.//}
> >  .else
> > +.if defined(PKG_SUFX)
> > +PKG_COMPRESSION_FORMAT?=     ${PKG_SUFX:S/.//}
> > +.warning PKG_SUFX is defined, if should be replace with PKG_COMPRESSION_FORMAT
> > +.endif
> > +PKG_SUFX=    .pkg
> > +.endif
>
> I love the idea here but can this be reverted for now please? Poudriere
> is not prepared for a default PKG_EXT change.
>
> 1. Now that it finds a new PKG_EXT of .pkg, it cannot find the existing
> bootstrap pkg.txz because it looks for Latest/pkg.pkg ->
> ../All/pkg-*.pkg even though the *existing* one is .txz. I'm not sure
> how this symlink is added before pkg is even built but it's not done in
> a compatible way.

I haven't seen an issue on my end with the symlinks for pkg which
seems to bootstrap ok and
I don't fully understand the problem statement.  I'd like to see what
bapt has to say for this one.

> 2. The new PKG_EXT makes Poudriere now find no existing packages so
> everything is rebuilt. Not a deal breaker in itself but a problem for
> future builds...

See below, maybe this is an acceptable transition hammer whenever
someone changes it.

> 3. The rebuild won't remove all the .txz files. I assume once `pkg repo`
> runs it will find both .txz and .pkg files and cause duplicate version
> problems in `pkg repo`

This one is poudriere specific, do you think an acceptable solution
would be to detect changes to the variable from the config versus the
repo by storing a canary in the repo and run the pkgclean module if it
changes?  I can try and do that tomorrow with your design approval or
guidance.


> I see something creates a .txz -> .pkg symlink. What deles that symlink
> if Poudriere only deletes/rebuilds the .pkg?

Unsure.

> ***
> "all those .txz entries are actually _duplicates_ in packagesite.yaml"
> seen on IRC just now by someone.
> I can confirm that looking in a packagesite.yaml I just generated. Both
> the older .txz version of a port and a new revision-bumped one appear in
> the yaml.
> ***
>
> 4. I've seen reports of packagesite.pkg which is concerning because
> Poudriere manages packagesite.txz with its own symlinks for
> hardlink-copied repositories. It does not use PKG_EXT there as they
> weren't packages, but I suspect we need special handling even with
> PKG_EXT added there.
>
> --
> Regards,
> Bryan Drewery
>