Questionable powerpc64 11.0-CURRENT assembler notation? missing @toc Assembler warnings from powerpc64-gcc tools...

Mon Mar 30 07:22:34 UTC 2015

Basic context:
> 
> # dmesg | head
> ...
> FreeBSD 11.0-CURRENT #0 r280598M: Fri Mar 27 18:26:08 PDT 2015
>     root at FBSDG5C0:/usr/obj/usr/src/sys/GENERIC64vtsc-NODEBUG powerpc
> gcc version 4.9.1 (FreeBSD Ports Collection for powerpc64) 
> ...

> # freebsd-version -ku; uname -apKU
> 11.0-CURRENT
> 11.0-CURRENT
> FreeBSD FBSDG5C0 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r280598M: Fri Mar 27 18:26:08 PDT 2015     root at FBSDG5C0:/usr/obj/usr/src/sys/GENERIC64vtsc-NODEBUG  powerpc powerpc64 1100066 1100066

Notably this buildworld/buildkernel is based on powerpc64-gcc from the powerpc64-xtoolchain-gcc. gcc 4.2.1 i missing from the system and is not built, nor has libstdc++ been built for buildworld/buildkernel use, nor clang. (For other experiments unrelated to buildworld/buildkernel I have recently installed lang/gcc5 and lang/clang36.) The C++ library is libc++, no libstdc++ use by buildworld/buildkernel.

The problem:

[Note: For all I know this could be a tools vintage issue where notations are not compatible across vintages. I report this mostly as an FYI.]

The assembler messages later are from a rebuild of this gcc 4.9.1 based powerpc64 11.0-CURRENT -r280598. I just happened to notice the messages in the build, they have been around in my activities before but they were not were I was focused.

I first quote the TOC_REF definition and some related material since the complaints are about some examples of TOC_REF usage (but not the ones in TOC_ENTRY's):

> #ifdef __powerpc64__
> #define TOC_REF(name)   __CONCAT(.L,name)
> #define TOC_ENTRY(name) \
>         .section ".toc","aw"; \
>         TOC_REF(name): \
>         .tc name[TC],name

In the below the lines with TOC_REF(...) notation are the lines being reported.

> /usr/src/sys/powerpc/aim/locore64.S: Assembler messages:
> /usr/src/sys/powerpc/aim/locore64.S:134: Warning: assuming @toc on symbol

>         /* Set up the stack pointer */
>         ld      %r1,TOC_REF(tmpstk)(%r2)
>         addi    %r1,%r1,TMPSTKSZ-96
>         add     %r1,%r1,%r31

> /usr/src/sys/powerpc/aim/trap_subr64.S:317: Warning: assuming @toc on symbol
> /usr/src/sys/powerpc/aim/trap_subr64.S:794: Warning: assuming @toc on symbol

> cpu_reset:
>         GET_TOCBASE(%r2)
> 
>         ld      %r1,TOC_REF(tmpstk)(%r2)        /* get new SP */
>         addi    %r1,%r1,(TMPSTKSZ-48)
> ...
> dbtrap:
>         /* Write the trap vector to SPRG3 by computing LR & 0xff00 */
>         mflr    %r1
>         andi.   %r1,%r1,0xff00
>         mtsprg3 %r1
>         
>         ld      %r1,TRAP_TOCBASE(0)             /* get new SP */
>         ld      %r1,TOC_REF(tmpstk)(%r1)
>         addi    %r1,%r1,(TMPSTKSZ-48)

> /usr/src/sys/powerpc/powerpc/swtch64.S: Assembler messages:
> /usr/src/sys/powerpc/powerpc/swtch64.S:149: Warning: assuming @toc on symbol

> cpu_switchin:
> #if defined(SMP) && defined(SCHED_ULE)
>         /* Wait for the new thread to become unblocked */
>         ld      %r6,TOC_REF(blocked_lock)(%r2)
> blocked_loop:
>         ld      %r7,TD_LOCK(%r13)
>         cmpd    %r6,%r7
>         beq-    blocked_loop
>         isync
> #endif

And more TOC_REF usage at the following that I do not show. Note that I have my own ofwcall64.S variant for helping to supply early-boot crash info if such happens so do not expect line numbers to carry over, possibly not even the count of warnings.

> /usr/src/sys/powerpc/ofw/ofwcall64.S: Assembler messages:
> /usr/src/sys/powerpc/ofw/ofwcall64.S:112: Warning: assuming @toc on symbol
> /usr/src/sys/powerpc/ofw/ofwcall64.S:116: Warning: assuming @toc on symbol
> /usr/src/sys/powerpc/ofw/ofwcall64.S:124: Warning: assuming @toc on symbol
> /usr/src/sys/powerpc/ofw/ofwcall64.S:211: Warning: assuming @toc on symbol
> /usr/src/sys/powerpc/ofw/ofwcall64.S:213: Warning: assuming @toc on symbol
> /usr/src/sys/powerpc/ofw/ofwcall64.S:267: Warning: assuming @toc on symbol
> /usr/src/sys/powerpc/ofw/ofwcall64.S:269: Warning: assuming @toc on symbol
> /usr/src/sys/powerpc/ofw/ofwcall64.S:276: Warning: assuming @toc on symbol

There are also some examples of:

> {standard input}: Assembler messages:
> {standard input}:5: Warning: unterminated string; newline inserted
> {standard input}:6: Warning: unterminated string; newline inserted

But that is not what this note is about and it would be better to report the details from a build without -j 8 to be certain what these messages go with.

Context details:

I originally bootstrapped to using powerpc64-xtoolchain-gcc's powerpc64-gcc using -r279514. After that I attempted rebuilding 279514 various times exploring what I could enable successfully vs. what I could not. -r280598 was my first attempt to svnlite upgrade then rebuild using just the powerpc64-gcc based cross tools environment. It required manually getting updated headers placed where it was looking but I was able to update.

The 4.2.1 gcc is not present. Nor has clang been built; 3.4.1 was removed by make delete-old after the initial bootstrap. I'm using powerpc64-gcc as the only system compiler after the bootstrap. For a time it was the only compiler present. No libcsdtc++ use for buildworld/buildkernel, just libc++ instead.

> make -j 8 CROSS_TOOLCHAIN=powerpc64-gcc \
> WITHOUT_CLANG_BOOTSTRAP= WITHOUT_CLANG= WITHOUT_CLANG_IS_CC= \
> WITHOUT_LLDB= \
> WITHOUT_GCC_BOOTSTRAP= WITHOUT_GCC= WITHOUT_GNUCXX= \
> WITHOUT_BOOT= WITHOUT_LIB32= \
> buildworld buildkernel \
> KERNCONF=GENERIC64vtsc-NODEBUG \
> TARGET=powerpc TARGET_ARCH=powerpc64

Notes: powerpc64-gcc does not allow WITH_BOOT= or WITH_LIB32= because -melf32ppc_fbsd is disallowed. As stands powerpc64-gcc is not sufficient context to build clang 3.6. Also the build environment ignores WITH_GCC_BOOTSTRAP= when XCC is a full path to a cross compiler. Apparently WITH_GCC= has a similar issue but for it I've not investigated where/how it is forcibly ignored.

For the below: Remember, no 4.2.1 gcc and no clang. So powerpc64-gcc also has to cover what those would normally be used for, not just the XCC, XCXX, XCPP usage.

> # more /etc/src.conf 
> NO_WERROR=
> WITH_LIBCPLUSPLUS=
> CC=/usr/local/bin/powerpc64-portbld-freebsd11.0-gcc
> CXX=/usr/local/bin/powerpc64-portbld-freebsd11.0-g++
> CPP=/usr/local/bin/powerpc64-portbld-freebsd11.0-cpp
> CROSS_BINUTILS_PREFIX=/usr/local/powerpc64-freebsd/bin/
> X_COMPILER_TYPE=gcc
> CFLAGS+=-L/usr/obj/usr/src/tmp/usr/lib/.
> # The order of the two CXXFLAGS additions below is important.
> CXXFLAGS+=-I/usr/obj/usr/src/tmp/usr/include/c++/v1/. -std=gnu++11 -L/usr/obj/usr/src/lib/libc++/.
> CXXFLAGS+=-I/usr/include/c++/v1/. -std=gnu++11 -L/usr/lib/.

(I make no claim that the above is a general solution but with some care about how to do builds it has allowed the experiments that I've done. I sometimes have used /usr/srcC/ instead of /usr/src/ : I have two 11.0-CURRENT source trees and tend to swap which I'm using at updates.)

> # more /etc/make.conf 
> WRKDIRPREFIX=/usr/obj/portswork
> #WITH_DEBUG=
> MALLOC_PRODUCTION=

> # svnlite st /usr/src --no-ignore
> ?       /usr/src/.snap
> ?       /usr/src/restoresymtable
> M       /usr/src/sys/ddb/db_main.c
> M       /usr/src/sys/ddb/db_script.c
> ?       /usr/src/sys/powerpc/conf/GENERIC64vtsc
> ?       /usr/src/sys/powerpc/conf/GENERIC64vtsc-NODEBUG
> ?       /usr/src/sys/powerpc/conf/GENERICvtsc
> ?       /usr/src/sys/powerpc/conf/GENERICvtsc-NODEBUG
> M       /usr/src/sys/powerpc/ofw/ofw_machdep.c
> M       /usr/src/sys/powerpc/ofw/ofwcall64.S

All of the above files that are relevant existed and were in use long before I started the powerpc64-xtoolchain-gcc related explorations. ofw_machdep.c is tied to a PowerMac-specific change for reliable booting. The db_'s and ofwcall64.S are tied to getting information from early-boot crashes if I get any more. The GENERIC*'s relate to that early-information gathering and to disabling ps3 so that I can then also have both vt and sc enabled.

Remember that gcc5 is not used for buildworld/buildkernel in the below and gcc5 was actually added after I'd done such builds...

> # find / \( -type d -name .svn -prune \) -or \( -name "libstdc++.*" -exec file {} \; \) | more
> /usr/local/lib/gcc5/libstdc++.so.6.0.21: ELF 64-bit MSB shared object, 64-bit PowerPC or cisco 7500, version 1 (FreeBSD), dynamically linked, not stripped
> /usr/local/lib/gcc5/libstdc++.so.6: symbolic link to libstdc++.so.6.0.21
> /usr/local/lib/gcc5/libstdc++.so: symbolic link to libstdc++.so.6.0.21
> /usr/local/lib/gcc5/libstdc++.a: current ar archive
> /usr/local/lib/gcc5/libstdc++.so.6.0.21-gdb.py: ASCII text
> /usr/obj/... (ignored here)
> /usr/lib/libstdc++.a: symbolic link to libc++.a
> /usr/lib/libstdc++.so: symbolic link to libc++.so

So buildworld and buildkernel here are based on libc++ to the exclusion of libstdc++.

There is more to getting things to this stage. For example since 11.0-CURRENT has CC:=gcc in csu/powerpc64/ I put in symbolic links to the 4.9.1 compiler. I'm not going to try to list everything here. Even the installation of powerpc64-gcc in a powerpc64 system requires manual intervention (last I did that anyway).

So far I've only dealt with the case of bootstrapping via powerpc64 self-hosting its own cross compile. More things have to be correct to actually cross compile from powerpc (non-64) and I've avoided dealing with the extra complications that are likely involved.

As this has been an exploration of the unfamiliar, I've had false starts, backtracking, and the like. While I've learned a bunch I doubt that I could start over and get to where I am by a step-by-step procedure that avoided backtracking, pondering, retries, and so on. In some respects I've been lucky that certain types of changes did not happen between my original and upgraded vintages: things would not have worked that did work.

I have not reached a stage of systematic procedure starting from a gcc 4.2.1 powerpc64 11.0-CURRENT but I have an existence proof of getting to this gcc 4.9.1 and libc++ based status.

===
Mark Millard
markmi at dsl-only.net