11.0-CURRENT: DTLS1_VERSION_MAJOR, /head/crypto/openssl/ssl/ssl_asn1.c -r280297 vs. updating to head snaphot -r280598

Fri Mar 27 12:50:41 UTC 2015

/head/crypto/openssl/ssl/ssl_asn1.c has a similar issue to /head/crypto/openssl/crypto/bio/bio_err.c  but relative to:

> /usr/local/bin/powerpc64-portbld-freebsd11.0-gcc -fpic -DPIC  -O2 -pipe   -DTERMIOS -DANSI_SOURCE -I/usr/src/secure/lib/libssl/../../../crypto/openssl -I/usr/src/secure/lib/libssl/../../../crypto/openssl/crypto -I/usr/obj/usr/src/secure/lib/libssl -DOPENSSL_THREADS -DDSO_DLFCN -DHAVE_DLFCN_H -std=gnu99 -fstack-protector -Wno-pointer-sign   -c /usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/ssl_asn1.c -o ssl_asn1.So
> /usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/ssl_asn1.c: In function 'd2i_SSL_SESSION':
> /usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/ssl_asn1.c:425:34: error: 'DTLS1_VERSION_MAJOR' undeclared (first use in this function)
>          || (ssl_version >> 8) == DTLS1_VERSION_MAJOR
>                                   ^

> # pwd
> /usr/src
> # find . \( -type d -name .svn -prune \) -or \( -type f -exec grep DTLS1_VERSION_MAJOR {} \; -print \) | more
> # define DTLS1_VERSION_MAJOR             0xFE
> ./crypto/openssl/ssl/dtls1.h
>         || (ssl_version >> 8) == DTLS1_VERSION_MAJOR
> ./crypto/openssl/ssl/ssl_asn1.c

> # find / \( -type d -name .svn -prune \) -or \( -type f -name dtls1.h -print \) | more
> /usr/src/crypto/openssl/ssl/dtls1.h
> /usr/obj/usr/src/tmp/usr/include/openssl/dtls1.h
> /usr/include/openssl/dtls1.h

> #include <stdio.h>
> #include <stdlib.h>
> #include "ssl_locl.h"
> #include <openssl/asn1_mac.h>
> #include <openssl/objects.h>
> #include <openssl/x509.h>

So the crypto/openssl/ssl/dtls1.h or /usr/obj/usr/src/tmp/usr/include/openssl/dtls1.h file is not directly included at all. Finding where it is included...

Omitting most Makefile lines...

> # find . \( -type d -name .svn -prune \) -or \( -type f -exec grep "dtls1\.h" {} \; -print \) | more
> INCS=   dtls1.h kssl.h srtp.h ssl.h ssl2.h ssl23.h ssl3.h tls1.h
> ./secure/lib/libssl/Makefile
> ...
> ./crypto/openssl/apps/Makefile
> /* ssl/dtls1.h */
> ./crypto/openssl/ssl/dtls1.h
> ...
> ./crypto/openssl/ssl/Makefile
> # include <openssl/dtls1.h>     /* Datagram TLS */
> ./crypto/openssl/ssl/ssl.h

crypto/openssl/ssl/ssl.h is not directly included. (I will stop here as far as the include sequence goes since a dtls1.h was found.)

The <openssl/dtls1.h> reference is finding /usr/include/openssl/dtls1.h (old) instead of /usr/obj/usr/src/tmp/usr/include/openssl/dtls1.h or /usr/src/crypto/openssl/ssl/dtls1.h (new):

> # diff -w /usr/include/openssl/dtls1.h /usr/obj/usr/src/tmp/usr/include/openssl/dtls1.h | more
> 87a88
> > # define DTLS1_VERSION_MAJOR             0xFE
> 123,129c124,128
> ...

===
Mark Millard
markmi at dsl-only.net

On 2015-Mar-27, at 02:44 AM, Mark Millard <markmi at dsl-only.net> wrote:

Basic context:

# freebsd-version -ku; uname -apKU
11.0-CURRENT
11.0-CURRENT
FreeBSD FBSDG5C0 11.0-CURRENT FreeBSD 11.0-CURRENT #1 r279514M: Sat Mar 21 05:15:23 PDT 2015     root at FBSDG5C0:/usr/obj/usr/srcC/sys/GENERIC64vtsc-NODEBUG  powerpc powerpc64 1100062 1100062

The problem:

Summary of the details that are listed later. All of the following exist:

> /usr/src/crypto/openssl/crypto/bio/bio.h
> /usr/obj/usr/src/tmp/usr/include/openssl/bio.h
> /usr/include/openssl/bio.h

The first two can be newer than the last during buildworld.

The buildworld compile of /head/crypto/openssl/crypto/bio/bio_err.c from an updated /usr/src can/does end up using the last instead of one of the first two, at least for the powerpc64-xtoolchain-gcc style of buildworld activity that I am trying.

The recent addition of BIO_F_DGRAM_SCTP_WRITE ends up with its definition missing because of this: during the build /usr/include/openssl/bio.h ends up being the file included and the compile fails from the missing additional definition.

Either the #include paths in /head/crypto/openssl/crypto/bio/bio_err.c or the command line arguments should force the /usr/obj/usr/src/tmp/usr/include/openssl/bio.h (or /usr/src/crypto/openssl/crypto/bio/bio.h ) vintage file to be found. The bio.h relevant includes are shown below...

> #include <stdio.h>
> #include <openssl/err.h>
> #include <openssl/bio.h>

More than bio.h might have such issues since there is an openssl/err.h include path in /head/crypto/openssl/crypto/bio/bio_err.c .

I have not checked for other .c files with similar issues for <openssl/...> usage during buildworld.

The problem details:

/head/crypto/openssl/crypto/bio/bio_err.c -r280297 added:

>     {ERR_FUNC(BIO_F_DGRAM_SCTP_WRITE), "DGRAM_SCTP_WRITE"},

and head (20150325 r280598) contains it.

But the BIO_F_DGRAM_SCTP_WRITE reference blocks buildworld (for at least /usr/local/bin/powerpc64-portbld-freebsd11.0-gcc (powerpc64-xtoolchain=gcc) use):

> /usr/local/bin/powerpc64-portbld-freebsd11.0-gcc -fpic -DPIC  -O2 -pipe   -DTERMIOS -DANSI_SOURCE -I/usr/src/secure/lib/libcrypto/../../../crypto/openssl -I/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto -I/usr/obj/usr/src/secure/lib/libcrypto -DOPENSSL_THREADS -DDSO_DLFCN -DHAVE_DLFCN_H -I/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/asn1 -I/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/evp -I/usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/modes -std=gnu89 -fstack-protector -Wno-pointer-sign   -c /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/bio/bio_err.c -o bio_err.So
> In file included from /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/bio/bio_err.c:63:0:
> /usr/src/secure/lib/libcrypto/../../../crypto/openssl/crypto/bio/bio_err.c:99:15: error: 'BIO_F_DGRAM_SCTP_WRITE' undeclared here (not in a function)
>     {ERR_FUNC(BIO_F_DGRAM_SCTP_WRITE), "DGRAM_SCTP_WRITE"},
>               ^

Looking to see where usage and definitions might be in /usr/src for -r280598 ...

> # pwd
> /usr/src
> $ find . \( -type d -name .svn -prune \) -or \( -type f -exec grep BIO_F_DGRAM_SCTP_WRITE {} \; -print \) | more
> # define BIO_F_DGRAM_SCTP_WRITE                           133
> ./crypto/openssl/crypto/bio/bio.h
>            BIOerr(BIO_F_DGRAM_SCTP_WRITE, ERR_R_MALLOC_ERROR);
> ./crypto/openssl/crypto/bio/bss_dgram.c
>    {ERR_FUNC(BIO_F_DGRAM_SCTP_WRITE), "DGRAM_SCTP_WRITE"},
> ./crypto/openssl/crypto/bio/bio_err.c

And looking at the list of includes in /head/crypto/openssl/crypto/bio/bio_err.c -r280297 shows:

> #include <stdio.h>
> #include <openssl/err.h>
> #include <openssl/bio.h>

That there was no complaint about bio.h being missing suggests that a <openssl/bio.h> was found but did not contain a BIO_F_DGRAM_SCTP_WRITE definition: so a different one than a copy of what the above find/grep reported.

Using a find to report other bio.h files shows:

> # find / \( -type d -name .svn -prune \) -or \( -type f -name bio.h -print \) | more
> /usr/src/crypto/openssl/crypto/bio/bio.h
> /usr/src/sys/sys/bio.h
> /usr/obj/usr/src/tmp/usr/include/openssl/bio.h
> /usr/include/openssl/bio.h
> /usr/include/sys/bio.h

(Ignoring .../sys/bio.h as distinct by content and by path prefix...)

The diff of /usr/obj/usr/src/tmp/usr/include/openssl/bio.h and /usr/include/openssl/bio.h shows the problem if the wrong file is found and used:

> diff -w /usr/src/crypto/openssl/crypto/bio/bio.h /usr/include/openssl/bio.h | less
> ...
> 797,798c775
> < /*
> <  * The following lines are auto generated by the script mkerr.pl. Any changes
> ---
>> /* The following lines are auto generated by the script mkerr.pl. Any changes
> 832d808
> < # define BIO_F_DGRAM_SCTP_WRITE                           133

Context details:

> make -j 8 CROSS_TOOLCHAIN=powerpc64-gcc 
> WITHOUT_CLANG_BOOTSTRAP= WITHOUT_CLANG= WITHOUT_CLANG_IS_CC= \
> WITHOUT_LLDB= \
> WITH_GCC_BOOTSTRAP= WITH_GCC= WITHOUT_GNUCXX= \
> WITHOUT_BOOT= WITHOUT_LIB32= \
> buildworld buildkernel \
> KERNCONF=GENERIC64vtsc-NODEBUG
> TARGET=powerpc TARGET_ARCH=powerpc64

> # svnlite info /usr/src
> Path: .
> Working Copy Root Path: /usr/src
> URL: https://svn0.us-west.freebsd.org/base/head
> Relative URL: ^/head
> Repository Root: https://svn0.us-west.freebsd.org/base
> Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f
> Revision: 280615
> Node Kind: directory
> Schedule: normal
> Last Changed Author: hselasky
> Last Changed Rev: 280598
> Last Changed Date: 2015-03-25 06:32:27 -0700 (Wed, 25 Mar 2015)

signals.h and pthread.h have been updated to more recent than -r280598 in order to avoid the __nonnull issues that exist as of -r280598.

> # svnlite st /usr/src --no-ignore
> ?       /usr/src/.snap
> ?       /usr/src/restoresymtable
> M       /usr/src/sys/ddb/db_main.c
> M       /usr/src/sys/ddb/db_script.c
> ?       /usr/src/sys/powerpc/conf/GENERIC64vtsc
> ?       /usr/src/sys/powerpc/conf/GENERIC64vtsc-NODEBUG
> ?       /usr/src/sys/powerpc/conf/GENERICvtsc
> ?       /usr/src/sys/powerpc/conf/GENERICvtsc-NODEBUG
> M       /usr/src/sys/powerpc/ofw/ofw_machdep.c
> M       /usr/src/sys/powerpc/ofw/ofwcall64.S

(The .c/.S changes are tied to a PowerMac-G5-specific boot-problem fix and getting information from early boot failures if I get any more. The GENERIC<?>'s remove ps3 in order to have both vt and sc at the same time.)

> # more /etc/src.conf 
> NO_WERROR=
> WITH_LIBCPLUSPLUS=
> CC=/usr/local/bin/powerpc64-portbld-freebsd11.0-gcc
> CXX=/usr/local/bin/powerpc64-portbld-freebsd11.0-g++
> CPP=/usr/local/bin/powerpc64-portbld-freebsd11.0-cpp
> CROSS_BINUTILS_PREFIX=/usr/local/powerpc64-freebsd/bin/
> X_COMPILER_TYPE=gcc
> CXXFLAGS+=-I/usr/obj/usr/srcC/tmp/usr/include/c++/v1/. -std=gnu++11 -L/usr/obj/usr/srcC/lib/libc++/.
> CXXFLAGS+=-I/usr/include/c++/v1/. -std=gnu++11 -L/usr/lib/.

(The above and just below experiments with mostly(?) avoiding use of gcc 4.2.1, even for CC/CXX/CPP contexts.)

> # ls -FPal /usr/bin/g[+c]*
> lrwxr-xr-x  1 root  wheel     48 Mar 20 02:03 /usr/bin/g++@ -> /usr/local/bin/powerpc64-portbld-freebsd11.0-g++
> lrwxr-xr-x  1 root  wheel     48 Mar 19 04:20 /usr/bin/gcc@ -> /usr/local/bin/powerpc64-portbld-freebsd11.0-gcc

> # more /etc/make.conf
> WRKDIRPREFIX=/usr/obj/portswork
> #WITH_DEBUG=
> MALLOC_PRODUCTION=

===
Mark Millard
markmi at dsl-only.net