build order race after SUBDIR_PARALLEL (264303)

[Luigi Rizzo]

Hi,
I have recently hit a problem when building stable/10 with
"make -j 8" which i tracked down to svn 264303, the
MFC of the SUBDIR_PARALLEL build feature.

With a large number of parallel tasks (and GCC;
it is a race condition so timing matters), the build fails
during the toolchain target with this:

--- sig_party.o ---
cc   -O2 -pipe  -I/usr/home/luigi/FreeBSD/R10/lib/libngatm/../../sys/contrib/ngatm -I/usr/home/luigi/FreeBSD/R10/../usr/obj-pico-amd64/usr/home/luigi/FreeBSD/R10/lib/libngatm
-I/usr/home/luigi/FreeBSD/R10/lib/libngatm/../../contrib/ngatm/libngatm -std=gnu99  -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -
Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline
-Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign -c /usr/home/luigi/FreeBSD/R10/lib/libngatm/../../sys/contrib/ngatm/netnatm/sig/sig_party.c -o sig_
party.o
--- all_subdir_libproc ---
/usr/home/luigi/FreeBSD/R10/../usr/obj-pico-amd64/usr/home/luigi/FreeBSD/R10/tmp/usr/bin/ld: cannot find -lsupc++
*** [libproc.so.2] Error code 1

make[5]: stopped in /usr/home/luigi/FreeBSD/R10/lib/libproc
1 error
...


Turns out that before SUBDIR_PARALLEL, libsupc++ was built
before libproc thus satisfying the dependency; but with
-j XXX we cannot guarantee the ordering and there is no
explicit constraint or .ORDER in the makefiles to build
things in the correct order.

This is a race condition so you may or may not see the problem
depending on what you are building and where.

I am seeing the problem consistently on stable/10 after 264303
when building toolchain with GCC (not clang) on an 8-core machine
and make -j 8

Building the above with -j 4 seem to work fine,
as it works building HEAD with -j 8

cheers
luigi