TODO/wish list for packaging the FreeBSD base system

[Glen Barber]

As previously mentioned, packaging the base system is a requirement of
the 11.0-RELEASE.

First, a point of clarification on the end goals that surround this.  At
present, there is *no* intent to connect pkg(8) to source-based upgrades
and/or 'installworld/installkernel'.  There are a few reasons for this,
the most notable reason is the implicit dependency on an active network
link and ability to bootstrap pkg(8) if it is not installed.  Since the
pkg(8) utility is not included in the base system, we cannot depend on
the binary to exist on a system, and enforcing network connectivity as
a requirement is at best a POLA violation.

The goal of this is to provide the ability to manage system installation
with pkg(8), not enforce its use as a requirement of FreeBSD systems.

The primary target audience of this work is mainly binary upgrade
consumers, while providing the scalability for everyone to be able to
manage their systems with pkg(8) without overhead and maintenance of
homemade tools.

Historically we have provided binary upgrade paths for the "default"
FreeBSD installation.  However, the current binary upgrade tools only
provide an upgrade path for x86-based systems (amd64 and i386).  Using
pkg(8) as for binary upgrades, we can expand beyond this, and provide an
upgrade path for non-x86 systems (arm, mips, powerpc).

Below follows a braindump of known issues, TODO tasks, wish list items,
etc.

Known issues:

- There are likely a number of other issues I have not yet encountered,
  so consider this list far from complete.

- plist prioritization: pkg(8) needs to be aware of package installation
  prioritization (ordering), which is one of the items bapt@ is aware
  of.  In particular, one of the major requirements of this is the
  ordering of rtld, libc, and libthr, which must be handled in that
  order.

- Conflicting packages are non-obvious until attempting to install or
  upgrade.  In a best case scenario, something like the FreeBSD-acct
  package may fail to install.  In a worst case scenario, this can be
  fatal to the system.  When FreeBSD-runtime fails to install, pkg(8)
  will rollback the changes, which behaviorally leads to removing all
  files included in the package - a dead system.  

- Several files/directories do not have an associated package.  The
  default package in this case is supposed to be 'FreeBSD-runtime',
  however there is a non-zero number of files/directories that have no
  package assignment.  A few examples are /etc/login.conf, /etc/rc,
  /etc/network.subr.  These can be found in the METALOG file, by looking
  for lines without 'tags=.*package=<foo>'.

- Files flagged for automatic merge that would normally be handled by
  tools like etcupdate(8) or mergemaster(8) do not appear to be handled
  properly.  In a recent build, I noticed /etc/devd/asus.conf.pkgnew was
  created (flagged as a file to handle merges in r279661), while the
  /etc/devd/asus.conf file did not have any local changes.  I suspect
  this is a bug, and still investigating.

- Some files installed during 'installworld' have the immutable file
  flag set as result of an 'afterinstall' target, which is not flagged
  as schg in the mtree(8) METALOG file.  This is one of the failure
  cases that can lead to a dead system, as mentioned above.  An ugly fix
  was committed as r294966 and r294972, however I am unhappy with this
  solution, since the intent of honoring MK_FSCHG=yes for the installed
  files is lost, as well as the installed files now being symbolic links
  instead of hard links.  I have tried fixing this with a pre-install
  script in the runtime.ucl file, but these attempts failed because
  test(1) does not recognize hard links.

TODO List / Wish List:

- Further separation of the base system (FreeBSD-runtime package) into
  more granular package sets.  Extreme caution needs to be taken when
  testing updates to do this, so using virtualization is encouraged.
  I have spent countless hours over the past few weeks recovering failed
  installations and/or dead systems as noted above.

- Creating meta-packages for logically-related utilities, and making the
  packages created more granular.  One example that was provided in
  private email was the 'FreeBSD-rcmds' package, where the use case
  given had no need for tools like ruptime(1), rwho(1), and the like
  were useful to have installed, but rcp(1), rsh(1), etc., were not
  necessary.

- Unless there is a specific reason to have an in-tree <package>.ucl
  file in src/release/packages/, I would like to see a template used
  instead.  Use cases where an in-tree <package>.ucl file is needed are
  runtime.ucl and kernel.ucl, where post-install scripts are defined.
  For the rest of the currently-existing package definitions, it is not
  necessary, and adds to the complexity of breaking down the base system
  into more granular package sets.  I tried a few tests yesterday using
  a file called '_template_.ucl', which if the release/packages/foo.ucl
  did not exist, the '_template_.ucl' file would be copied to the stage
  directory.  This did not work, however, because there can be several
  '<package>.ucl' files for any given 'package', notably runtime.plist,
  debug.plist (which should be runtime-debug.plist), runtime-lib32.plist
  and so on.

Glen

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-pkgbase/attachments/20160129/b0d9567b/attachment.sig>