Total confusion over toolchain/xdev behavior

Wed Jul 9 03:48:51 UTC 2014

On Jul 8, 2014, at 6:06 PM, Baptiste Daroussin <bapt at FreeBSD.org> wrote:

> On Mon, Jul 07, 2014 at 07:29:01PM -0600, Warner Losh wrote:
>> 
>> On Jul 7, 2014, at 5:52 PM, Baptiste Daroussin <bapt at FreeBSD.org> wrote:
>> 
>>> On Mon, Jul 07, 2014 at 05:27:25PM -0600, Warner Losh wrote:
>>>> 
>>>> On Jul 7, 2014, at 2:51 PM, Ian Lepore <ian at FreeBSD.org> wrote:
>>>> 
>>>>> On Sun, 2014-07-06 at 16:07 -0700, Sean Bruno wrote:
>>>>>> Objective:  install an xcompile toolchain into a jail for use by
>>>>>> poudriere during arm/mips/sparc/power ports pkgs builds.  The build
>>>>>> should be possible from a non-root user.
>>>>>> 
>>>>>> As far as I can tell, the xdev target is completely busted for non-clang
>>>>>> arch's right now as it tries to build clang no matter what I do.  Its
>>>>>> missing some pretty key documentation to making it work correctly, so a
>>>>>> lot of my attempts have been "guess and check" with verbose make.
>>>>>> 
>>>>>> -----------------------------------------------------------------------
>>>>>> Attempt #1:
>>>>>> I have been trying non-root xdev builds:
>>>>>> MAKEOBJDIRPREFIX=/var/tmp make -s -j8 xdev XDEV=mips XDEV_ARCH=mips
>>>>>> -- dies because it tries to compile CLANG.
>>>>>> -----------------------------------------------------------------------
>>>>>> 
>>>>>> Attempt #2:
>>>>>> Apply a hack from Baptiste that isn't quite right, but at least gets
>>>>>> farther, note the missing variable causing "//usr/mips-freebsd"
>>>>>> http://people.freebsd.org/~sbruno/src.ops.mk.diff
>>>>>> 
>>>>>> ===> gnu/usr.bin/cc/gcov (all)
>>>>>> mtree populating //usr/mips-freebsd
>>>>>> mkdir: //usr/mips-freebsd: Permission denied
>>>>>> *** Error code 1
>>>>>> -----------------------------------------------------------------------
>>>>>> 
>>>>>> Attempt #3: Add XDTP
>>>>>> MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=mips XDEV_ARCH=mips
>>>>>> XDTP=/var/tmp/mips_cc
>>>>>> 
>>>>>> Try defining a XDTP=/var/tmp/mips_cc with the above patch applied, get's
>>>>>> a bit farther but compile failure in locating critical include files.
>>>>>> 
>>>>>> ===> gnu/lib/libstdc++ (obj,depend,all,install)
>>>>>> In file included from /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc
>>>>>> ++/../../../contrib/libstdc++/src/bitmap_allocator.cc:30:
>>>>>> /home/sbruno/bsd/fbsd_head/gnu/lib/libstdc++/../../../contrib/libstdc
>>>>>> ++/include/ext/bitmap_allocator.h:37:54: error: cstddef: No such file or
>>>>>> directory
>>>>>> -----------------------------------------------------------------------
>>>>>> 
>>>>>> Attempt #4: Add the additional XDDESTDIR
>>>>>> MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=mips XDEV_ARCH=mips
>>>>>> XDTP=/var/tmp/mips_cc XDESTDIR=/var/tmp/mips_dst
>>>>>> -- Same results as attempt #3
>>>>>> -----------------------------------------------------------------------
>>>>>> 
>>>>>> Even attempting to do stuff for *clang* enabled architectures bails
>>>>>> because its not respecting prefixes:
>>>>>> MAKEOBJDIRPREFIX=/var/tmp make -s -j 8 xdev XDEV=arm XDEV_ARCH=armv6
>>>>>> -- bails because it tries to:
>>>>>> ===> usr.bin/clang/tblgen (all)
>>>>>> mtree populating //usr/armv6-freebsd
>>>>>> mtree: etc/ntp: Permission denied
>>>>>> _xi-cross-tools
>>>>>> ===> xdev gnu/usr.bin/binutils (install)
>>>>>> ===> gnu/usr.bin/binutils/libiberty (install)
>>>>>> ===> gnu/usr.bin/binutils/libbfd (install)
>>>>>> ===> gnu/usr.bin/binutils/libopcodes (install)
>>>>>> ===> gnu/usr.bin/binutils/libbinutils (install)
>>>>>> ===> gnu/usr.bin/binutils/addr2line (install)
>>>>>> ===> gnu/usr.bin/binutils/as (install)
>>>>>> ===> gnu/usr.bin/binutils/ld (install)
>>>>>> install: //usr/armv6-freebsd/usr/bin/ld: Permission denied
>>>>>> *** Error code 71
>>>>>> 
>>>>>> -----------------------------------------------------------------------
>>>>>> Adding XDTP and XDDESTDIR results in a little more progress but obvious
>>>>>> failures to attempt and install things directly into my host system:
>>>>>> 
>>>>>> MAKEOBJDIRPREFIX=/var/tmp make -s xdev XDEV=arm XDEV_ARCH=armv6
>>>>>> XDDESTDIR=/var/tmp/arm_cc XDTP=/var/tmp/armv6_cc
>>>>>> ===> secure/lib/libssh (install)
>>>>>> ===> usr.bin/lex/lib (obj,depend,all,install)
>>>>>> mkdir: ../../../../usr: Permission denied
>>>>>> *** Error code 1
>>>>>> 
>>>>>> Stop.
>>>>>> make[1]: stopped in /home/sbruno/bsd/fbsd_head
>>>>>> *** Error code 1
>>>>>> 
>>>>>> Stop.
>>>>>> make: stopped in /home/sbruno/bsd/fbsd_head
>>>>> 
>>>>> It looks to me like the permission part of the problem is being caused
>>>>> by a lack of DESTDIR=.  Without that, it's trying to install to /usr and
>>>>> you don't have permission for that.  Maybe the confusion is because the
>>>>> xdev target inherently builds-and-installs, unlike most other targets
>>>>> that separate those two actions.
>>>> 
>>>> OK. After some detective work, it looks like libstdc++ needs to be done before libsupc++ is done. I’ve added this dependency in r268377 and was able to do a full xdev build with a clean obj dir:
>>>> 
>>>> 	rm -rf $HOME/F $MAKEOBJDIRPREFIX/mips-freebsd
>>>> 	mkdir $HOME/F
>>>> 	make xdev DESTDIR=$HOME/F XDEV=mips XDEV_ARCH=mips WITHOUT_CLANG=t WITHOUT_CLANG_BOOTSTRAP=t WITH_GCC=t WITH_GCC_BOOTSTRAP=t WITH_GNUCXX=t -j 20
>>> 
>>> We can avoid most of the above by using a patch like the following:
>>> http://people.freebsd.org/~bapt/Makefile.inc1.diff
>>> Extending the same thing xi-cross-tools and xb-cross-tools (expect the
>>> WITH_GNUCXX=t because it it not set in src.opts.mk when it imho should.)
>> 
>> The patch looks good to my eye. Did you want me to expand it, or do you want to
>> do the honors?
>> 
>> About the rest… Yea, you may be right….  MK_GNUCXX is an odd duck, and that’s
>> likely the problem that should be fixed in a different way. It is really an internal
>> variable that should be set based on the actual compiler type (possibly with an
>> override for the odd-duck pair of clang and libstdc++ which may not be worth
>> supporting). It is telling us we’re doing something horribly wrong and we should listen
>> to that rather than add another compiler-related kludge to the build system.  I’ll work
>> on that bit.
>> 
>> Also an aside: The horribly long command line was to try to get to the bottom of
>> the breakage, not to promote it as the proper way of doing things.
> 
> I haven't committed because it isn't complete yet :)
> I have updated it to the following:
> http://people.freebsd.org/~bapt/Makefile.inc1.diff
> but it is not enough as I end up with:
> 
> /tmp/tmips//usr/mips64-freebsd/usr/bin/ld: cannot find -lsupc++
> 
> So it goes further but it is not yet enough

I actually think xdev and a couple others should move to Makefile, and the sub targets stay in Makefile.inc and be invoked with the right TARGET and TARGET_ARCH so that we get the right defaults.

Warner
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20140708/a55e79a5/attachment.sig>