building an 8.4-STABLE i386 poudriere jail on an 10.0-STABLE amd64 host

[Glen Barber]

On Sat, Aug 30, 2014 at 12:20:46AM -0700, Don Lewis wrote:
> On 29 Aug, Glen Barber wrote:
> > The 8.4-RELEASE was built on a machine running 9-STABLE, so make(1),
> > gcc(1), and so on were not that far diverged for there to be any real
> > issues (such as what you're seeing).
> 
> Interestingly enough, make and clang don't seem to be an issue so far as
> I can tell.

Behavior in make(1) between fmake(1) and bmake(1) have been subtle
enough over the transition, but in a longer jump (i.e., 8-STABLE to
10-STABLE), it is actually quite significant.

Clang is an entirely different situation, as you have seen from the
libstdc++.  This is not entirely fault of the clang(1) switch, however;
it is effectively the behavior of the build was broken prior to the
switch, at least, as far as I understand it.  That is, ar(1) files were
being used from the host environment, instead from outside of .OBJDIR.

> I do have the fmake port installed because poudriere told
> me that it was necessary, but if I just cd to the directory containing
> the source for 8-STABLE, "make buildworld" mostly seems to work.  The
> incompatibility of the patch -b option was the first problem that I ran
> into, and that just required a few individual Makefile edits.  The
> second problem was the lex upgrade during sometime after 10 was
> branched, I worked around that by making lex a bootstrap tool using the
> existing knob.  Yacc was already a bootstrap tool.
> 
> The final problem that is vexing me is that the cross-tool version of
> lint.  The source for lint is basically unchanged between 8-STABLE and
> 10-STABLE.  The only significant change that I noticed was a one line
> patch   to lint1/tree.c (and lint1 is the SIGBUS victim) documented
> here: <https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=152549> and
> fixed in r224702.  I applied that patch and saw no change.
> 
> If I run the cross version of lint with the -B option to get it to use
> the host version of lint1, then things work.  It's just the cross
> version of lint1 that is broken.  Both the host and cross versions of
> lint1 are compiled with clang, from nearly identical source code, though
> the cross version was built with a parser and lexer gernerated using the
> bootstrap versions of lex and yacc.  What's really maddening is that I
> don't see anything wrong if I point gdb at the core file.  The next
> thing I'm going to try is to try to run lint1 under gdb and single step
> it.  That's tricky because lint1 is run from lint and I need to figure
> out how to set up its arguments and input file.
> 
> Oh, and I discovered that neither "truss -fa" nor ktrace output anything
> for exec*() calls.
>  
> > The snapshot builder in use now, at that time was running 10-CURRENT.
> > Several "hacks" were in place, such as WITH_GNUCXX (which somehow turned
> > into 4 or more src.conf(5) entries by now), which would allow building
> > 8.x on 10-CURRENT to somewhat work.  That did not last long, though.
> 
> Oh yeah ... I stumbled across that problem as well.  I think I only
> needed WITH_GCC and WITH_GNUCXX.
> 

In that case, I was mostly pointing out the differences between "just
two branches."  It actually gets much worse than that.

For ARM builds, on 11-CURRENT, for example, here's what is exported:

 export XDEV_FLAGS="WITH_GCC=1 WITH_GCC_BOOTSTRAP=1 WITHOUT_CLANG_IS_CC=1"
 
Compared to 10-STABLE:

 export XDEV_FLAGS="WITH_GCC=1 WITH_GNUCXX=1 WITHOUT_CLANG_IS_CC=1"

I'm not suggesting this is the problem in your case, however, just
illustrating how things can differ between two otherwise-similar
branches can have a huge impact.

Somewhat related, I am heavily considering adding WITH_SCRABBLE=1 knob,
which will try to guess the intended result is based on arbitrary alpha
matches in the defined() pattern.  I am about 40% joking, but the other
60% of me thinks it is more sane than some of the things we have in the
build toolchain now.  :-)

> > I think it was around when bmake became the default in -CURRENT that
> > I stopped spending time trying to build 8.x snapshots -- at that point,
> > the time spent debugging the build failures and updating the scripts to
> > "fix" the build became increasingly less beneficial.
> > 
> > Now, having said all that, I have a possible solution that, quite
> > honestly, may or may not work (I have no idea at this point).
> > 
> > You can grab the 8.4-RELEASE distribution set from FTP, and extract it
> > into a scratch directory, check out the stable/8/ tree into the scratch
> > directory usr/src/, then do:
> > 
> >     # chroot /scratch make -C /usr/src buildworld [...]
> > 
> > That would at least provide you with the binary bits you want for your
> > package builds.
> 
> I've actually got a couple of 8-STABLE boxes here already.  They are
> just underpowered for mass building packages, which is why I'm not just
> running "poudriere bulk" on one of them.  The other missing piece is
> going from there to a working poudriere jail while not using the "jail
> -c" command.
> 
> Hmn ... maybe I could build the jail on one of my 8.4-STABLE machines
> and just copy the jail bits over ...
> 

This is where my suggestion about 'bootstrapping' the build jail comes
in; if you get the 8.x userland in place, you can check out the src/
tree within it, chroot(8) to the 8.x userland, and then do the normal
buildworld/installworld dance.

Or, if you are a ZFS consumer, 'zfs send | zfs recv' sounds like an
equally good solution.

> > I *think* this is somewhat what the cluster package builders do, but
> > then again, I do not know off-hand if packages for 8.x are build for
> > pkg(8) - they may be only pkg_install(1).
> 
> I think they are building pkg(8) packages, but they just use
> 8.4-RELEASE, plus security updates I imagine.  I've been able to do
> that here for testing individual package builds on 8.x.
> 

I think you may be right about the pkg(8) conversion for 8.x.  I am
getting myself confused with the exp-builders versus the production
pkg(8) builders, which yes, they do use the -RELEASE (and subsequent
patchsets).

Glen

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20140830/84a86661/attachment.sig>