CROSS_TOOLCHAIN=powerpc64-gcc mishandles "Substitution Failure Is Not An Error" when compiling clang and stops the build

Mon Mar 23 17:26:45 UTC 2015

https://llvm.org/bugs/show_bug.cgi?id=22771 from Dimitry Andric's submittal of the problem indicates that the libc++ code is not a "Substitution Failure Is Not An Error" context and so is wrong. (C is certainly simpler than C++ for identifying what applies where.)

They have an improvement but Richard Smith's tiny test case shows it is not yet correct:

> Here's a testcase that fails with Clang:
> 
>   #define __has_feature(x) 0
>   #include <type_traits>
>   class X { X(const X&); };
>   bool b = std::is_convertible<const X&, X>::value;
> 
> (Using a public deleted copy constructor fails similarly.)

in that both the original code and the improvement fail to compile the above but instead treat it as an error. (Dimitry Andric tested the improvement and https://llvm.org/bugs/show_bug.cgi?id=22771 shows the error that he got.)

===
Mark Millard
markmi at dsl-only.net

On 2015-Mar-20, at 11:27 PM, Mark Millard <markmi at dsl-only.net> wrote:

Basic context:

> # dmesg | head
> ...
> FreeBSD 11.0-CURRENT #0 r279514M: Wed Mar 18 20:11:15 PDT 2015
>    root at FBSDG5C0:/usr/obj/usr/srcC/sys/GENERIC64vtsc-NODEBUG powerpc
> gcc version 4.9.1 (FreeBSD Ports Collection for powerpc64) 
> ...

> # freebsd-version -ku; uname -apKU
> 11.0-CURRENT
> 11.0-CURRENT
> FreeBSD FBSDG5C0 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r279514M: Wed Mar 18 20:11:15 PDT 2015     root at FBSDG5C0:/usr/obj/usr/srcC/sys/GENERIC64vtsc-NODEBUG  powerpc powerpc64 1100062 1100062

> make -j 8 CROSS_TOOLCHAIN=powerpc64-gcc \
> WITHOUT_CLANG_BOOTSTRAP= WITH_CLANG= WITH_CLANG_IS_CC= \
> WITH_LLDB= \
> WITH_GCC_BOOTSTRAP= WITH_GCC= WITHOUT_GNUCXX= \
> WITHOUT_BOOT= WITHOUT_LIB32= \
> buildworld buildkernel \
> KERNCONF=GENERIC64vtsc-NODEBUG \
> TARGET=powerpc TARGET_ARCH=powerpc64

For the context set by:

> # more /etc/src.conf
> CC=/usr/local/bin/powerpc64-portbld-freebsd11.0-gcc
> CXX=/usr/local/bin/powerpc64-portbld-freebsd11.0-g++
> CPP=/usr/local/bin/powerpc64-portbld-freebsd11.0-cpp
> CROSS_BINUTILS_PREFIX=/usr/local/powerpc64-freebsd/bin/
> X_COMPILER_TYPE=gcc
> WITH_LIBCPLUSPLUS=
> #
> # CXXFLAGS For buildworld/buildkernel CROSS_TOOLCHAIN=powerpc64-gcc use...
> # spans being-built and (failing finding those directories) live and so for
> # -DNO_CLEAN after being-built ones are in place depends on self-hodsting
> # where the two are sufficient compatibile.
> #
> # I've used .../. paths so I can tell use of these from other sources of paths.
> #
> # Actually only appropriate for for _includes _libraries _depend everything build32 :
> CXXFLAGS+=-I/usr/obj/usr/srcC/tmp/usr/include/c++/v1/. -std=gnu++11 -L/usr/obj/usr/srcC/lib/libc++/.
> #
> # Actually only appropriate for for _worldtmp _legacy _bootstrap-tools _cleanobj _obj _build-tools _cross-tools :
> CXXFLAGS+=-I/usr/include/c++/v1/. -std=gnu++11 -L/usr/lib/.
> #
> # But for self-hosting in a cross tools like manor sometimes having both can work.
> #
> NO_WERROR=

The problem:

(Somewhat reformatted text...)

> In file included from /usr/include/c++/v1/./algorithm:625:0,
>                 from /usr/srcC/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/StringRef.h:13,
>                 from /usr/srcC/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/StringMap.h:17,
>                 from /usr/srcC/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/Support/SpecialCaseList.h:51,
>                 from /usr/srcC/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/SpecialCaseList.cpp:17:
> /usr/include/c++/v1/./type_traits:
> 
> In instantiation of 'struct std::__1::__is_convertible<const llvm::StringMap<llvm::SpecialCaseList::Entry>&, llvm::StringMap<llvm::SpecialCaseList::Entry>, 0u, 0u>':
> /usr/include/c++/v1/./type_traits:943:62:
> required from
> 
> 'struct std::__1::is_convertible<const llvm::StringMap<llvm::SpecialCaseList::Entry>&, llvm::StringMap<llvm::SpecialCaseList::Entry> >'
> /usr/include/c++/v1/./utility:269:77:
> required by
> 
> substitution of 'template<class _U1, class _U2> std::__1::pair<_T1, _T2>::pair(const std::__1::pair<_U1, _U2>&, typename std::__1::enable_if<(std::__1::is_convertible<const _U1&, _T1>::value && std::__1::is_convertible<const _U2&, _T2>::value)>::type*) [with _U1 = llvm::StringRef; _U2 = llvm::StringMap<llvm::SpecialCaseList::Entry>]'
> /usr/srcC/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/StringMap.h:371:55:
> required from
> 
> 'llvm::StringMap<ValueTy, AllocatorTy>::MapEntryTy& llvm::StringMap<ValueTy, AllocatorTy>::GetOrCreateValue(llvm::StringRef, InitTy) [with InitTy = llvm::StringMap<llvm::SpecialCaseList::Entry>; ValueTy = llvm::StringMap<llvm::SpecialCaseList::Entry>; AllocatorTy = llvm::MallocAllocator; llvm::StringMap<ValueTy, AllocatorTy>::MapEntryTy = llvm::StringMapEntry<llvm::StringMap<llvm::SpecialCaseList::Entry> >]'
> /usr/srcC/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/StringMap.h:375:43:
> required from
> 
> 'llvm::StringMap<ValueTy, AllocatorTy>::MapEntryTy& llvm::StringMap<ValueTy, AllocatorTy>::GetOrCreateValue(llvm::StringRef) [with ValueTy = llvm::StringMap<llvm::SpecialCaseList::Entry>; AllocatorTy = llvm::MallocAllocator; llvm::StringMap<ValueTy, AllocatorTy>::MapEntryTy = llvm::StringMapEntry<llvm::StringMap<llvm::SpecialCaseList::Entry> >]'
> /usr/srcC/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/StringMap.h:299:32:
> required from
> 
> 'ValueTy& llvm::StringMap<ValueTy, AllocatorTy>::operator[](llvm::StringRef) [with ValueTy = llvm::StringMap<llvm::SpecialCaseList::Entry>; AllocatorTy = llvm::MallocAllocator]'
> /usr/srcC/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/SpecialCaseList.cpp:120:21:
> required from
> 
> here
> /usr/include/c++/v1/./type_traits:881:87: error: use of deleted function 'llvm::StringMap<llvm::SpecialCaseList::Entry>::StringMap(const llvm::StringMap<llvm::SpecialCaseList::Entry>&)'
>         sizeof(__is_convertible_imp::__test<_T2>(__is_convertible_imp::__source<_T1>())) == 1
>                                                                                       ^

and a little more...

> In file included from /usr/srcC/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/Support/SpecialCaseList.h:51:0,
>                 from /usr/srcC/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/SpecialCaseList.cpp:17:
> /usr/srcC/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/StringMap.h:226:7:
> note: 'llvm::StringMap<llvm::SpecialCaseList::Entry>::StringMap(const llvm::StringMap<llvm::SpecialCaseList::Entry>&)' is implicitly declared as deleted because 'llvm::StringMap<llvm::SpecialCaseList::Entry>' declares a move constructor or move assignment operator
> class StringMap : public StringMapImpl {
>       ^

where...

> namespace __is_convertible_imp
> {
> template <class _Tp> char  __test(_Tp);
> template <class _Tp> __two __test(...);
> #ifndef _LIBCPP_HAS_NO_RVALUE_REFERENCES
> template <class _Tp> _Tp&& __source();
> #else             
> template <class _Tp> typename remove_reference<_Tp>::type& __source();
> #endif
> 
> ...
> }

In other words:

> __is_convertible_imp::__source<const llvm::StringMap<llvm::SpecialCaseList::Entry>&>()

is a function returning llvm::StringMap<llvm::SpecialCaseList::Entry>&& (an r-value reference, not a universal one). (I'm presuming rvalue references but the overall point is the same even for const llvm::StringMap<llvm::SpecialCaseList::Entry>& as the return type.)

As for __is_convertible_imp::__test:

> __is_convertible_imp::__test<llvm::StringMap<llvm::SpecialCaseList::Entry>>
>  (llvm::StringMap<llvm::SpecialCaseList::Entry>)

is a function returning a value of type char that is used when the argument can be passed into a llvm::StringMap<llvm::SpecialCaseList::Entry> type of parameter. Failure of this to be possible is not of itself an error ("substitution failure is not an error" for selecting template functions): it just means that other example definitions of __is_convertible_imp::__test functions may be used to match the argument instead.

and

> __is_convertible_imp::__test<llvm::StringMap<llvm::SpecialCaseList::Entry>>(...)

is a function returning a value of type __two for all potential, valid argument lists. (_T2 is actually ignored.) When both __is_convertible_imp::__test's are non-errors the prior one is picked by the language rules: a better parameter vs. argument match.

Note that the status of deleted copy constructors can contribute to if

> __is_convertible_imp::__test<llvm::StringMap<llvm::SpecialCaseList::Entry>>
>  (llvm::StringMap<llvm::SpecialCaseList::Entry>)

is a match for the convertibility classifications and it is not an error for them to do so.

The size of the selected __is_convertible_imp::__test<llvm::StringMap<llvm::SpecialCaseList::Entry>>'s result type indicates the is-convertible status (sizeof(char) != sizeof(__two)) and the sizeof use means that overall the expression is a constexpr (compile time constant) that is based on the type analysis by the compiler, with no actual constructions happening.

But the powerpc64-gcc 4.9.1 compiler is not applying the principle of "substitution failure is not an error" and is not using the implicitly deleted-status to cause a mis-match for:

> __is_convertible_imp::__test<llvm::StringMap<llvm::SpecialCaseList::Entry>>
>  (llvm::StringMap<llvm::SpecialCaseList::Entry>)

So the compiler rejects the code for supposed use of an implicitly deleted StringMap(const StringMap &RHS) constructor. This is a wrong classification for the code.

It appears that powerpc64-gcc (gcc 4.9.1) is not up to compiling 11.0-CURRENT -r279514's clang/llvm as-is.

Context details, mostly applying to both gcc 4.2.1 based world/kernel with powperpc-gcc in use for cross compiling and gcc 4.9.1 based world/kernel:

powerpc64-gcc automatically looks in /usr/local/include for header files (like most such ports) and this can pick up the wrong file(s). I ended up renaming

/usr/local/include/iconv.h

so that it would not be found and used where a file with different content from my /usr/srcC/... was needed.

CROSS_TOOLCHAIN=powerpc64-gcc use does not allow -melf32ppc_fbsd for WITH_BOOT= and WITH_LIB32= build activity to use. Even if there was a powerpc-xtoolchain-gcc (and powerpc-gcc) around then selectively doing just the WITH_BOOT= and WITH_LIB32= would still not seem a natural fit.

In my experiments at times for the 4.9.1 based live-world I've placed the following symbolic links:

> # ls -FPal /usr/lib/libstdc*
> lrwxr-xr-x  1 root  wheel  8 Mar 19 03:47 /usr/lib/libstdc++.a@ -> libc++.a
> lrwxr-xr-x  1 root  wheel  9 Mar 19 03:47 /usr/lib/libstdc++.so@ -> libc++.so
> 
> # ls -FPal /usr/bin/gcc /usr/bin/g++
> lrwxr-xr-x  1 root  wheel  48 Mar 19 04:20 /usr/bin/g++@ -> /usr/local/bin/powerpc64-portbld-freebsd11.0-g++
> lrwxr-xr-x  1 root  wheel  48 Mar 19 04:20 /usr/bin/gcc@ -> /usr/local/bin/powerpc64-portbld-freebsd11.0-gcc

For example csu/powerpc64/... uses "gcc" directly, ignoring XCC and CC.

The CC, CXX, and CPP in...

> # more /etc/src.conf
> CC=/usr/local/bin/powerpc64-portbld-freebsd11.0-gcc
> CXX=/usr/local/bin/powerpc64-portbld-freebsd11.0-g++
> CPP=/usr/local/bin/powerpc64-portbld-freebsd11.0-cpp
> ...

that essentially duplicate the XCC, XCXX, XCPP assignments from CROSS_TOOLCHAIN=powerpc64-gcc are there because otherwise various stages do not use the cross tools (legacy, bootstrap-tools, build-tools, cross-tools, kernel-tools, lib32's build-tools). Originally I was investigating how far I could get without involving gcc 4.2.1 and what would be involved.

In the running world built-via-powerpc64-gcc environment /etc/src.conf having ...

> # CXXFLAGS For buildworld/buildkernel CROSS_TOOLCHAIN=powerpc64-gcc use...
> CXXFLAGS+=-I/usr/obj/usr/srcC/tmp/usr/include/c++/v1/. -std=gnu+=11 -L/usr/obj/usr/srcC/lib/libc++/.

is used because otherwise (for example)...

> /usr/local/bin/powerpc64-portbld-freebsd11.0-g++ -fpic -DPIC  -O2 -pipe -DHAVE_CONFIG_H -I/usr/srcC/contrib/atf -I/usr/srcC/lib/atf/libatf-c++/../libatf-c -I. -DHAVE_CONFIG_H -fstack-protector -Wsyste
> m-headers -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wpointer-arith -Wno-uninitialized   -c /usr/srcC/contrib/atf/atf-c++/detail/application.cpp -o application.So

ends up with...

> In file included from /usr/srcC/contrib/atf/atf-c++/detail/application.cpp:26:0:
> /usr/srcC/contrib/atf/atf-c++/detail/application.hpp:29:19: fatal error: ostream: No such file or directory
> #include <ostream>
>                 ^
> compilation terminated.

from lack of -I/usr/obj/usr/srcC/tmp/usr/include/c++/v1/. in CXXFLAGS and the like. This is despite the /usr/srcC/Makefile.inc1 logic (that ends up inactive/ineffective for some reason):

> .if ${XCC:M/*}
> ...
> .if defined(X_COMPILER_TYPE) && ${X_COMPILER_TYPE} == gcc
> XCFLAGS+=       -isystem ${WORLDTMP}/usr/include -L${WORLDTMP}/usr/lib
> XCXXFLAGS+=     -I${WORLDTMP}/usr/include/c++/v1 -std=gnu++11 -L${WORLDTMP}/../lib/libc++
> DEPFLAGS+=      -I${WORLDTMP}/usr/include/c++/v1

As for /etc/make.conf it looks like...

> # more /etc/make.conf
> #CC=/usr/local/bin/powerpc64-portbld-freebsd11.0-gcc
> #CXX=/usr/local/bin/powerpc64-portbld-freebsd11.0-g++
> #CPP=/usr/local/bin/powerpc64-portbld-freebsd11.0-cpp
> #CROSS_BINUTILS_PREFIX=/usr/local/powerpc64-freebsd/bin/
> #X_COMPILER_TYPE=gcc
> # CXFLAGS For normal powerpc64-gcc use...
> # AVOID during the buildworld/buildkernel activities:
> #     _includes _libraries _depend everything build32.
> # See /etc/src.conf for example buildworld/buildkernel
> # values for that context.
> #CXXFLAGS+=-I/usr/include/c++/v1 -std=gnu++11 -L/usr/obj/usr/srcC/lib/libc++
> #
> #CC=/usr/local/bin/gcc5
> #CXX=/usr/local/bin/g++5
> #CPP=/usr/local/bin/cpp5
> #CC=/usr/local/bin/clang36
> #CXX=/usr/local/bin/clang++36
> #CPP=/usr/local/bin/clang-cpp36
> WRKDIRPREFIX=/usr/obj/portswork
> #WITH_DEBUG=
> MALLOC_PRODUCTION=

(Some of the comments show how gcc5/g++5 use was temporarily forced while rebuilding powerpc64-gcc from a booted world that had been built based on powerpc64-gcc in a gcc 4.2.1 world.)

> # svnlite info /usr/srcC/
> Path: /usr/srcC
> Working Copy Root Path: /usr/srcC
> URL: https://svn0.us-west.freebsd.org/base/head
> Relative URL: ^/head
> Repository Root: https://svn0.us-west.freebsd.org/base
> Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f
> Revision: 279514
> Node Kind: directory
> Schedule: normal
> Last Changed Author: adrian
> Last Changed Rev: 279514
> Last Changed Date: 2015-03-01 18:27:25 -0800 (Sun, 01 Mar 2015)

> # svnlite status /usr/srcC/ --no-ignore
> ?       /usr/srcC/.snap
> M       /usr/srcC/contrib/llvm/include/llvm/ADT/IntrusiveRefCntPtr.h
> ?       /usr/srcC/restoresymtable
> M       /usr/srcC/sys/ddb/db_main.c
> M       /usr/srcC/sys/ddb/db_script.c
> ?       /usr/srcC/sys/powerpc/conf/GENERIC64vtsc
> ?       /usr/srcC/sys/powerpc/conf/GENERIC64vtsc-NODEBUG
> ?       /usr/srcC/sys/powerpc/conf/GENERICvtsc
> ?       /usr/srcC/sys/powerpc/conf/GENERICvtsc-NODEBUG
> M       /usr/srcC/sys/powerpc/ofw/ofw_machdep.c
> M       /usr/srcC/sys/powerpc/ofw/ofwcall64.S

IntrusiveRefCntPtr.h does not matter if clang building is not involved. It needed a friend declaration to gain access to a private field, otherwise compilation stopped.

And the rest existed in my environment before I started this powerpc64-xtoolchain-gcc exploration.

lib/libnv/test/dnv_tests.cc and lib/libnv/test/nv_tests.cc are from later (-r279760) than the rest of the unmodified source code. This is in order to remove ambiguous overload issues.

> # svnlite info /usr/ports
> Path: /usr/ports
> Working Copy Root Path: /usr/ports
> URL: https://svn0.us-west.freebsd.org/ports/head
> Relative URL: ^/head
> Repository Root: https://svn0.us-west.freebsd.org/ports
> Repository UUID: 35697150-7ecd-e111-bb59-0022644237b5
> Revision: 381120
> Node Kind: directory
> Schedule: normal
> Last Changed Author: dbn
> Last Changed Rev: 381120
> Last Changed Date: 2015-03-12 10:13:39 -0700 (Thu, 12 Mar 2015)

I have gcc5 and clang36 ports installed. I've made no use of clang36.

On a powerpc64 11.0-CURRENT powerpc64-xtoolchain-gcc fails to complete its installation because powerpc64-gcc fails to complete its installation. The problems were 4 mismatched file names and one file also not put into staging. I copied appropriate files to the missing names and place and from that status the installation was able to continue and complete via postmaster with -C (as long as powerpc64-gcc was not in use rebuilding itself: some other compiler was in use for that activity instead).

===
Mark Millard
markmi at dsl-only.net