Using python meta ports as dependencies and tree woes

[Marcus von Appen]

Dear all,

after creating quite some fuss with the update yesterday, most issues
could be fixed with pulling in lang/python as run-time and build
dependency on a temporary basis.

The good thing is that everything should work at the moment. The bad
thing is that we have a lot of ports

a) being not compatible with different python versions, but using
   ${PREFIX}/bin/python
b) suffering from cross-port dependency issues, which cause some
   dependency of the ports to be built pulling in a python version,
   which the port implicitly expects at build and run-time.

Let's look at a) first. www/node requires the fixed python version 2.7
at build time. Within its build infrastructure it uses
${PREFIX}/bin/python, though, expecting the system that this python
version will be the correct one.

Users specifying python3.x as default won't be able to build that port
at the moment. Several other ports suffer from a similar problem, either
at build or run-time, making it impossible (at the moment), to use a
python version different from our pre-defined default (2.7) properly.

Several ports allow us to override the python version, either by
defining --with-python (for many autotools configure scripts) or setting
a PYTHON environment variable. Several do not, but use `python`
hard-coded in their files or use shebang lines such as
`#!/usr/bin/env python`.

Those ports need to be fixed to properly pull in the correct python
version. Many of them can be fixed quite easily by adding post-patch
targets, specifying CONFIGURE_ARGS or setting a MAKE_ENV variable to use
the correct ${PYTHON_CMD}.

This is a necessary task to be done and requires us to test each port,
which adds python as dependency via USE_PYTHON.

b) is a bit more complicated. A prominent example, that broke several
ports at once, is x11-toolkits/gtk20. It installs a script
/usr/local/bin/gtk-builder-convert, which uses the
`#!/usr/bin/env python` shebang. gtk20 did not define a run-time
dependency on python, though. This caused several gnome ports to fail on
build, since lang/python was not pulled in as dependency and hence no
${PREFIX}/bin/python could be used by the script. The whole construct
worked, because some other port added lang/python27 before, so that this
flaw remained undiscovered.

If we want to support different python versions for package building, we
have to take the bitter pill and check the ports for those
cross-dependency issues. This basically means lang/python may only be
added as dependency by a port, if it is guaranteed that the port works
with _every_ python version.

Both variations can be tested more or less easily. The last hack I added
yesterday, was to add lang/python in bsd.python.mk, so that it is
guaranteed to be around now.
Removing it from bsd.python.mk should reveal those issues again. On
testing, it must be assured, that no port uses lang/python directly, so
that it is not added as dependency and thus neither bin/python nor
bin/python2 or bin/python3 are around.

Those both parts are high priority tasks on my agenda before moving on
with everything else. It will be time consuming, though, since we have
to inform maintainers about those changes prior to get the commit
approval. The status quo of pulling in lang/python can be preserved a
the same time, since the change can be tested locally and should not
have any impact on the upstream ports tree.

How would the testing look like?

1. Remove the lang/python dependency from bsd.python.mk in a local ports
   tree.
2. Inspect, build and install the port to check
3. If it fails to build or run, fix it
4. Report the fix to the maintainer (and to the upstream developers,
   where possible)
5. Get the commit approval, sanity-test on the upstream ports tree
6. Commit

I think, we also should let maintainers know on ports@ about that, with
a brief guide to test their ports for those flaws, so they can assist in
testing and fixing their own ports. Maybe we can even get a redports
build server with the necessary modifications, so we and the maintainers
can test the ports.

Cheers
Marcus
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-python/attachments/20130819/2a78761f/attachment.sig>