11.0-CURRENT powerpc64 cont'd: without adding to LIBADD linking various programs are getting undefined references

Mon Nov 30 06:11:08 UTC 2015

Note: After the below examples rescue had a much larger list of Undefined References from various sources. At least for now I've given up and not tried to track down what to add to get rescue to link.

Explicit md references were used to avoid symbol problems in:

/usr/src/cddl/sbin/zfs/Makefile
/usr/src/cddl/sbin/zpool/Makefile
/usr/src/cddl/usr.bin/zinject/Makefile
/usr/src/cddl/usr.bin/zstreamdump/Makefile
/usr/src/cddl/usr.bin/ztest/Makefile
/usr/src/cddl/usr.sbin/zdb/Makefile
/usr/src/cddl/usr.sbin/zhack/Makefile

(A separate submittal covered the first few of those in more detail.)

libcrypto use had similar issues, for example:

> Index: /usr/src/sbin/atm/atmconfig/Makefile
> ===================================================================
> --- /usr/src/sbin/atm/atmconfig/Makefile        (revision 291443)
> +++ /usr/src/sbin/atm/atmconfig/Makefile        (working copy)
> @@ -17,7 +17,7 @@
>  .if !defined(RESCUE) && ${MK_BSNMP} != "no"
>  CFLAGS+=       -DWITH_BSNMP
>  SRCS+= oid.h atmconfig_device.c
> -LIBADD+=       bsnmp
> +LIBADD+=       crypto bsnmp
>  . if ${MK_DYNAMICROOT} == "no" && ${MK_OPENSSL} != "no"
>  LIBADD+=       crypto
>  . endif

Without the addition the result was:

> --- all_subdir_sbin ---
> /usr/local/bin/powerpc64-portbld-freebsd11.0-gcc -O2 -pipe -isystem /usr/obj/usr/src/tmp/usr/include/. -L/usr/obj/usr/src/tmp/usr/lib/. -L/usr/obj/usr/src/tmp/li
> b/. -I/usr/obj/usr/src/sbin/atm/atmconfig -DWITH_BSNMP -g -std=gnu99 -fstack-protector-strong -Wsystem-headers -Wall -Wno-format-y2k -W -Wno-unused-parameter -Ws
> trict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscrip
> ts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wno-pointer-sign  -L/usr/obj/usr/src/tmp/usr/lib/. -L/usr/obj/usr/src/tmp/lib/. -L/usr/obj
> /usr/src/lib/libc++/. -L/usr/lib/. -o atmconfig.full main.o diag.o natm.o atmconfig_device.o   -lbsnmp
. . .
> --- all_subdir_sbin ---
> /usr/local/bin/powerpc64-freebsd-ld: warning: libcrypto.so.8, needed by /usr/obj/usr/src/tmp/usr/lib/./libbsnmp.so, not found (try using -rpath or -rpath-link)
> /usr/obj/usr/src/tmp/usr/lib/./libbsnmp.so: undefined reference to `EVP_md5'
> /usr/obj/usr/src/tmp/usr/lib/./libbsnmp.so: undefined reference to `EVP_sha1'
> /usr/obj/usr/src/tmp/usr/lib/./libbsnmp.so: undefined reference to `EVP_DecryptInit'
> /usr/obj/usr/src/tmp/usr/lib/./libbsnmp.so: undefined reference to `EVP_DigestInit'
> /usr/obj/usr/src/tmp/usr/lib/./libbsnmp.so: undefined reference to `EVP_EncryptFinal'
> /usr/obj/usr/src/tmp/usr/lib/./libbsnmp.so: undefined reference to `EVP_EncryptUpdate'
> /usr/obj/usr/src/tmp/usr/lib/./libbsnmp.so: undefined reference to `EVP_DigestUpdate'
> /usr/obj/usr/src/tmp/usr/lib/./libbsnmp.so: undefined reference to `EVP_CIPHER_CTX_set_padding'
> /usr/obj/usr/src/tmp/usr/lib/./libbsnmp.so: undefined reference to `EVP_DigestFinal'
> /usr/obj/usr/src/tmp/usr/lib/./libbsnmp.so: undefined reference to `EVP_DecryptFinal'
> /usr/obj/usr/src/tmp/usr/lib/./libbsnmp.so: undefined reference to `EVP_aes_128_cfb128'
> /usr/obj/usr/src/tmp/usr/lib/./libbsnmp.so: undefined reference to `EVP_DecryptUpdate'
> /usr/obj/usr/src/tmp/usr/lib/./libbsnmp.so: undefined reference to `EVP_des_cbc'
> /usr/obj/usr/src/tmp/usr/lib/./libbsnmp.so: undefined reference to `EVP_EncryptInit'
> /usr/obj/usr/src/tmp/usr/lib/./libbsnmp.so: undefined reference to `EVP_MD_CTX_cleanup'
> /usr/obj/usr/src/tmp/usr/lib/./libbsnmp.so: undefined reference to `EVP_CIPHER_CTX_cleanup'
> collect2: error: ld returned 1 exit status
> *** [atmconfig.full] Error code 1

The -L's ending with /.'s are from my /etc/src.conf . They are associated with trying to self-build on a powerpc64 PowerMac using only powerpc64-xtoolchain-gcc's powerpc64-gcc WITH_LIBCPLUSPLUS: powerpc64-gcc is acting as both the host toolchain and the CROSS_TOOLCHAIN. No 4.2.1 gcc is present. I know this is not a usual way of doing things. There may be better ways. But if I manage to expose some missing dependency tracking some good came of it.

And there was the need to add sbuf explicitly for its indirect use, such as for:

> Index: /usr/src/sbin/iscontrol/Makefile
> ===================================================================
> --- /usr/src/sbin/iscontrol/Makefile    (revision 291443)
> +++ /usr/src/sbin/iscontrol/Makefile    (working copy)
> @@ -2,7 +2,7 @@
>  
>  SRCS= iscontrol.c pdu.c fsm.c config.c login.c auth_subr.c misc.c
>  PROG= iscontrol
> -LIBADD=        cam md
> +LIBADD=        sbuf cam md
>  S= ${.CURDIR}/../../sys
>  
>  WARNS?=        3

Otherwise I got:

> /usr/local/bin/powerpc64-portbld-freebsd11.0-gcc -O2 -pipe -isystem /usr/obj/usr/src/tmp/usr/include/. -L/usr/obj/usr/src/tmp/usr/lib/. -L/usr/obj/usr/src/tmp/li
> b/. -I/usr/src/sbin/iscontrol/../../sys -g -std=gnu99 -fstack-protector-strong -Wsystem-headers -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototype
> s -Wmissing-prototypes -Wpointer-arith -Wno-uninitialized -Wno-pointer-sign  -L/usr/obj/usr/src/tmp/usr/lib/. -L/usr/obj/usr/src/tmp/lib/. -L/usr/obj/usr/src/lib
> /libc++/. -L/usr/lib/. -o iscontrol.full iscontrol.o pdu.o fsm.o config.o login.o auth_subr.o misc.o   -lcam  -lmd
. . .
> --- all_subdir_sbin ---
> /usr/obj/usr/src/tmp/usr/lib/./libcam.so: undefined reference to `sbuf_hexdump at FBSD_1.4'
> collect2: error: ld returned 1 exit status

Then there was both ssl and crypto for fetch:

> Index: /usr/src/usr.bin/fetch/Makefile
> ===================================================================
> --- /usr/src/usr.bin/fetch/Makefile     (revision 291443)
> +++ /usr/src/usr.bin/fetch/Makefile     (working copy)
> @@ -2,6 +2,6 @@
>  
>  PROG=          fetch
>  CSTD?=         c99
> -LIBADD=                fetch
> +LIBADD=                ssl crypto fetch
>  
>  .include <bsd.prog.mk>

without which I got:

> --- all_subdir_usr.bin ---
> /usr/local/bin/powerpc64-portbld-freebsd11.0-gcc -O2 -pipe -isystem /usr/obj/usr/src/tmp/usr/include/. -L/usr/obj/usr/src/tmp/usr/lib/. -L/usr/obj/usr/src/tmp/li
> b/. -g -std=iso9899:1999 -fstack-protector-strong -Wsystem-headers -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpoin
> ter-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-dec
> ls -Wold-style-definition -Wno-pointer-sign  -L/usr/obj/usr/src/tmp/usr/lib/. -L/usr/obj/usr/src/tmp/lib/. -L/usr/obj/usr/src/lib/libc++/. -L/usr/lib/. -o fetch.
> full fetch.o   -lfetch
. . .
> --- all_subdir_usr.bin ---
> /usr/local/bin/powerpc64-freebsd-ld: warning: libssl.so.8, needed by /usr/obj/usr/src/tmp/usr/lib/./libfetch.so, not found (try using -rpath or -rpath-link)
> /usr/local/bin/powerpc64-freebsd-ld: warning: libcrypto.so.8, needed by /usr/obj/usr/src/tmp/usr/lib/./libfetch.so, not found (try using -rpath or -rpath-link)
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `X509_LOOKUP_file'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `ASN1_STRING_length'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_get_error'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `CRYPTO_free'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_write'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSLv23_client_method'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `sk_value'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `MD5_Final'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_free'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `X509_STORE_add_lookup'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_CTX_use_certificate_chain_file'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_set_fd'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `X509_NAME_ENTRY_get_data'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_CTX_set_verify'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `X509_NAME_oneline'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `ASN1_STRING_data'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `X509_get_subject_name'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `X509_get_issuer_name'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_get_version'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_library_init'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_read'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `X509_STORE_CTX_get_current_cert'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_get_current_cipher'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_CTX_ctrl'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_new'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_set_connect_state'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_shutdown'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `GENERAL_NAMES_free'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_CTX_new'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_get_peer_certificate'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `X509_NAME_get_entry'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_CTX_load_verify_locations'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `X509_get_ext_d2i'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `X509_load_crl_file'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_ctrl'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `X509_STORE_set_flags'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_CTX_use_PrivateKey_file'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `X509_free'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `X509_NAME_get_index_by_NID'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `sk_num'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_connect'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_CTX_free'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_load_error_strings'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_CTX_get_cert_store'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `MD5_Init'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `SSL_CIPHER_get_name'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `ERR_print_errors_fp'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `MD5_Update'
> /usr/obj/usr/src/tmp/usr/lib/./libfetch.so: undefined reference to `ASN1_STRING_to_UTF8'
> collect2: error: ld returned 1 exit status

The make commands were:

> make -j 6 -DNO_CLEAN WITH_FAST_DEPEND= CROSS_TOOLCHAIN=powerpc64-gcc WITH_LIBCPLUSPLUS= WITHOUT_CLANG_BOOTSTRAP= WITH_CLANG= WITH_CLANG_IS_CC= WITH_LLDB= WITHOUT_GCC_BOOTSTRAP= WITHOUT_GCC= WITHOUT_GNUCXX= WITHOUT_BOOT= WITHOUT_LIB32= buildworld buildkernel KERNCONF=GENERIC64vtsc-NODEBUG TARGET=powerpc TARGET_ARCH=powerpc64

(The NO_CLEAN is from after the first build problem and is to avoid long rebuild times to get back to the same point.)

Context details if you care:

> # freebsd-version -ku; uname -aKU
> 11.0-CURRENT
> 11.0-CURRENT
> FreeBSD FBSDG5C0 11.0-CURRENT FreeBSD 11.0-CURRENT #12 r281630M: Sat Apr 18 01:15:13 PDT 2015     root at FBSDG5C0:/usr/obj/usr/src/sys/GENERIC64vtsc-NODEBUG  powerpc 1100070 1100070

I am attempting an update after having been away from the PowerMacs for months.

> # svnlite info /usr/src
> Path: /usr/src
> Working Copy Root Path: /usr/src
> URL: https://svn0.us-west.freebsd.org/base/head
> Relative URL: ^/head
> Repository Root: https://svn0.us-west.freebsd.org/base
> Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f
> Revision: 291443
> Node Kind: directory
> Schedule: normal
> Last Changed Author: nwhitehorn
> Last Changed Rev: 291442
> Last Changed Date: 2015-11-28 23:16:08 -0800 (Sat, 28 Nov 2015)

> # more /etc/src.conf
> NO_WERROR=
> WITH_DEBUG=
> WITH_DEBUG_FILES=
> MALLOC_PRODUCTION=
> CC=/usr/local/bin/powerpc64-portbld-freebsd11.0-gcc
> CXX=/usr/local/bin/powerpc64-portbld-freebsd11.0-g++
> CPP=/usr/local/bin/powerpc64-portbld-freebsd11.0-cpp
> CROSS_BINUTILS_PREFIX=/usr/local/powerpc64-freebsd/bin/
> X_COMPILER_TYPE=gcc
> DEPFLAGS+=-isystem /usr/obj/usr/src/tmp/usr/include/. -I/usr/obj/usr/src/tmp/usr/include/c++/v1/. -I/usr/include/c++/v1/.
> L/usr/obj/usr/src/tmp/usr/lib/lib32/.
> CFLAGS+=-isystem /usr/obj/usr/src/tmp/usr/include/. -L/usr/obj/usr/src/tmp/usr/lib/. -L/usr/obj/usr/src/tmp/lib/.
> LDFLAGS+=-L/usr/obj/usr/src/tmp/usr/lib/. -L/usr/obj/usr/src/tmp/lib/.
> CXXFLAGS+=-isystem /usr/obj/usr/src/tmp/usr/include/. -I/usr/obj/usr/src/tmp/usr/include/c++/v1/. -std=gnu++11 -L/usr/obj/usr/src/lib/libc++/.
> LDFLAGS+=-L/usr/obj/usr/src/lib/libc++/.
> CXXFLAGS+=-I/usr/include/c++/v1/. -std=gnu++11 -L/usr/lib/.
> LDFLAGS+=-L/usr/lib/.

> # svnlite info /usr/ports/
> Path: /usr/ports
> Working Copy Root Path: /usr/ports
> URL: https://svn0.us-west.freebsd.org/ports/head
> Relative URL: ^/head
> Repository Root: https://svn0.us-west.freebsd.org/ports
> Repository UUID: 35697150-7ecd-e111-bb59-0022644237b5
> Revision: 402562
> Node Kind: directory
> Schedule: normal
> Last Changed Author: rene
> Last Changed Rev: 402562
> Last Changed Date: 2015-11-28 15:08:03 -0800 (Sat, 28 Nov 2015)

===
Mark Millard
markmi at dsl-only.net