poudriere builds all dependencies, even if packages have already been built

Wed Apr 3 11:26:50 UTC 2013

Am 03.04.2013 um 12:53 schrieb Bryan Drewery:

> On 4/3/2013 4:17 AM, Stefan Bethke wrote:
>> Firstly, is there a better mailing list to discuss poudriere?
>> 
>> I'm just getting my feet wet with poudriere, and are a bit surprised that all dependencies for a port are built, instead of installing the packages that got built earlier already.  Is this intentional?  Is there a configuration option to have the requisite packages installed before building a port?
>> 
>> 
>> Thanks,
>> Stefan
>> 
> 
> Here or pkg@ is fine.
> 
> This is exactly what it does. It builds the dependencies one time then
> installs then builds the port needing them. On the next run if the
> dependencies have not updated it reuses them.

That's what I would expect, but that's not what I'm seeing.

I've been running build incrementally, fixing a problem, then running bulk again.  As expected, only the packages that hadn't been built successfully yet were built.  However, dependencies for these ports are not installed from the already created packages, but are recompiled again.  In this build log, you can see that pkg is installed from the port instead of from the package; I seem to get this for a large number of builds.  I wouldn't really mind except for the enormous amount of time it takes to rebuild the same ports over and over again.

I've had some trouble with poudriere not mounting things where they belong.  I manually created /usr/ports and /wrkdirs and recreated the "clean" snapshot.  This worked around the "mount: no such file or directory" problems, but I guess there might be an underlying cause to these and a couple more issues.  How would I go about identifying these?  I think I followed the installation instructions very carefully.

$ head -100 ImageMagick-6.8.0.7_1.log 
build started at Wed Apr  3 07:31:08 UTC 2013
port directory: /usr/ports/graphics/ImageMagick
building for: 9.1-STABLE amd64
maintained by: c.petrik.sosa at gmail.com
Makefile ident:      $FreeBSD: head/graphics/ImageMagick/Makefile 315309 2013-03-26 15:55:59Z bapt $
Poudriere version: 2.4.1
---Begin Environment---
BLOCKSIZE=K
MAIL=/var/mail/root
STATUS=1
PKG_EXT=txz
FORCE_PACKAGE=yes
tpid=97139
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/root/bin
POUDRIERE_BUILD_TYPE=bulk
NBPARALLEL=4
PKGNG=1
PKG_DELETE=/usr/local/sbin/pkg delete -y -f
PKG_ADD=/usr/local/sbin/pkg add
PWD=/root/eisenboot
LOGS=/var/poudriere/data/logs
HOME=/root
USER=root
SKIPSANITY=0
LOCALBASE=/usr/local
PACKAGE_BUILDING=yes
VERSION=2.4.1
---End Environment---

---Begin make.conf---
#### /root/eisenboot/poudriere.d/make.conf ####
WITH_PKGNG=yes
# Das sollte eigentlich poudriere selber machen
WRKDIRPREFIX?=	/wrkdirs
WRKDIR?=		${WRKDIRPREFIX}${.CURDIR:S/${PORTSDIR}//}
MASTER_SITE_BACKUP?=	\
	http://mirror.hanse.de/pub/FreeBSD/ports/distfiles/${DIST_SUBDIR}/
MASTER_SITE_OVERRIDE?=	${MASTER_SITE_BACKUP}

# Port-Konfiguration
openvpn_UNSET=	EASYRSA
PACKAGE_BUILDING=yes
DISABLE_MAKE_JOBS=yes
---End make.conf---

---Begin OPTIONS List---
===> The following configuration options are available for ImageMagick-6.8.0.7_1:
     16BIT_PIXEL=on: 16bit pixel support
     BZIP2=on: bzip2 compression support
     DJVU=off: DJVU format support (needs THREADS)
     DOCS=on: Build and/or install documentation
     FFTW=on: Discrete Fourier Transform support
     FONTCONFIG=on: X11 font configuration support
     FPX=on: FlashPix image format support
     FREETYPE=on: TrueType font rendering support
     GRAPHVIZ=off: Graphviz support
     GSLIB=off: libgs (Postscript SHLIB) support
     HDRI=off: High dynamic range images support
     JASPER=on: JPEG 2000 support via JasPer
     JBIG=on: JBIG image format support
     JPEG=on: JPEG image format support
     LCMS2=on: LCMS2 support
     LQR=on: Liquid Rescale support
     LZMA=on: LZMA compression support
     MODULES=on: Modules support
     OPENEXR=off: HDR image format support via OpenEXR
     OPENMP=off: Parallel processing support via OpenMP
     PANGO=off: Pango rendering library support
     PDF=on: PDF document support
     PERL=on: Perl scripting language support
     PNG=on: PNG image format support
     SVG=on: SVG vector image format support
     TESTS=off: Run bundled self-tests after build
     THREADS=off: Threading support
     TIFF=on: TIFF image format support
     WEBP=on: WebP image format support
     WMF=on: Windows Metafile support
===> Use 'make config' to modify these settings
---End OPTIONS List---
=======================<phase: depends      >==========================
===>   ImageMagick-6.8.0.7_1 depends on file: /usr/local/sbin/pkg - not found
===>    Verifying install for /usr/local/sbin/pkg in /usr/ports/ports-mgmt/pkg
===>  License BSD accepted by the user
===> Fetching all distfiles required by pkg-1.0.11 for building
===>  Extracting for pkg-1.0.11
===>  License BSD accepted by the user
===> Fetching all distfiles required by pkg-1.0.11 for building
=> SHA256 Checksum OK for pkg-1.0.11.tar.xz.
===>  Patching for pkg-1.0.11
===>  Configuring for pkg-1.0.11
===>  Building for pkg-1.0.11
sed -e 's,%%PKGVERSION%%,1.0.11,' Doxyfile.in > Doxyfile
===> external (all)
===> external/sqlite (all)
Warning: Object directory not changed from original /wrkdirs/ports-mgmt/pkg/pkg-1.0.11/external/sqlite
cc  -O2 -pipe -fno-strict-aliasing  -fPIC -DHAVE_READLINE=1  -I/usr/include/edit -DSQLITE_OMIT_AUTOVACUUM  -DSQLITE_OMIT_BLOB_LITERAL  -DSQLITE_OMIT_DECLTYPE  -DSQLITE_OMIT_EXPLAIN  -DSQLITE_OMIT_DEPRECATED  -DSQLITE_OMIT_LOAD_EXTENSION  -DSQLITE_OMIT_PROGRESS_CALLBACK  -DSQLITE_OMIT_TCL_VARIABLE  -DSQLITE_OMIT_UTF16  -DSQLITE_OMIT_CAT  -DSQLITE_OMIT_CHECK  -DSQLITE_OMIT_AUTOINIT  -DSQLITE_OMIT_COMPILEOPTION_DIAGS  -DSQLITE_OMIT_INTEGRITY_CHECK  -DSQLITE_OMIT_BUILTIN_TEST  -DSQLITE_OMIT_SHARED_CACHE  -DUSE_PREAD  -DSQLITE_THREADSAFE=1  -DSQLITE_TEMP_STORE=3  -Dmain=sqlite3_shell  -DNDEBUG -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wno-pointer-sign -Wformat=2 -Wno-format-extra-args -Werror -c sqlite3.c -o sqlite3.o
cc  -O2 -pipe -fno-strict-aliasing  -fPIC -DHAVE_READLINE=1  -I/usr/include/edit -DSQLITE_OMIT_AUTOVACUUM  -DSQLITE_OMIT_BLOB_LITERAL  -DSQLITE_OMIT_DECLTYPE  -DSQLITE_OMIT_EXPLAIN  -DSQLITE_OMIT_DEPRECATED  -DSQLITE_OMIT_LOAD_EXTENSION  -DSQLITE_OMIT_PROGRESS_CALLBACK  -DSQLITE_OMIT_TCL_VARIABLE  -DSQLITE_OMIT_UTF16  -DSQLITE_OMIT_CAT  -DSQLITE_OMIT_CHECK  -DSQLITE_OMIT_AUTOINIT  -DSQLITE_OMIT_COMPILEOPTION_DIAGS  -DSQLITE_OMIT_INTEGRITY_CHECK  -DSQLITE_OMIT_BUILTIN_TEST  -DSQLITE_OMIT_SHARED_CACHE  -DUSE_PREAD  -DSQLITE_THREADSAFE=1  -DSQLITE_TEMP_STORE=3  -Dmain=sqlite3_shell  -DNDEBUG -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wno-pointer-sign -Wformat=2 -Wno-format-extra-args -Werror -c shell.c -o shell.o
building static sqlite3 library
ranlib libsqlite3.a
===> external/libyaml (all)

-- 
Stefan Bethke <stb at lassitu.de>   Fon +49 151 14070811