Fwd: 11.0-CURRENT: SBUF_INCLUDENUL, /head/sys/kern/subr_sbuf.c -r280193 vs. updating to head snaphot -r280598

Fri Mar 27 16:32:46 UTC 2015

About the below Ian Lepore wrote:

> This and the other similar reports on current@ appear to be problems
> with the xtoolchain ports, not the base build system, and probably
> should have been reported to the port's maintainer, or on ports at .  Or
> perhaps it's some sort of usage error, I don't know anything about the
> xtoolchain stuff.  In any case, there doesn't seem to be anything wrong
> with the base build using the supported build mechanisms.

===
Mark Millard
markmi at dsl-only.net

Begin forwarded message:

Subject: 11.0-CURRENT: SBUF_INCLUDENUL, /head/sys/kern/subr_sbuf.c -r280193 vs. updating to head snaphot -r280598
From: Mark Millard <markmi at dsl-only.net>
Date: 2015-March-26 at 09:15:46 PM PDT
Cc: FreeBSD PowerPC ML <freebsd-ppc at freebsd.org>
To: freebsd-current at freebsd.org, Ian Lepore <ian at FreeBSD.org>

Basic context:

# freebsd-version -ku; uname -apKU
11.0-CURRENT
11.0-CURRENT
FreeBSD FBSDG5C0 11.0-CURRENT FreeBSD 11.0-CURRENT #1 r279514M: Sat Mar 21 05:15:23 PDT 2015     root at FBSDG5C0:/usr/obj/usr/srcC/sys/GENERIC64vtsc-NODEBUG  powerpc powerpc64 1100062 1100062

The problem:

Summary of the details that are listed later. Both of the following exist:

/usr/src/sys/sys/sbuf.h
/usr/include/sys/sbuf.h

The first can be newer than the 2nd during buildworld.

The buildworld compile of /head/sys/kern/subr_sbuf.c from an updated /usr/src can/does end up using the second instead of the first, at least for the powerpc64-xtoolchain-gcc style of buildworld activity that I am trying.

The recent addition of SBUF_INCLUDENUL use ends up with its definition missing because of this: during the build /usr/include/sys/sbuf.h ends up being the file included and the compile fails from the missing additional definition.

Either the #include paths in /head/sys/kern/subr_sbuf.c or the command line arguments should force the /usr/src/sys/sys/sbuf.h vintage file to be found. The /head/sys/kern/subr_sbuf.c relevant includes are shown below...

> #include <sys/cdefs.h>
> __FBSDID("$FreeBSD: head/sys/kern/subr_sbuf.c 280193 2015-03-17 21:00:31Z ian $");
> 
> #include <sys/param.h>
> 
> #ifdef _KERNEL
> #include <sys/ctype.h>
> #include <sys/errno.h>
> #include <sys/kernel.h>
> #include <sys/malloc.h>
> #include <sys/systm.h>
> #include <sys/uio.h>
> #include <machine/stdarg.h>
> #else /* _KERNEL */
> #include <ctype.h>
> #include <errno.h>
> #include <stdarg.h>
> #include <stdio.h>
> #include <stdlib.h>
> #include <string.h>
> #endif /* _KERNEL */
> 
> #include <sys/sbuf.h>

I have not checked for other .c files with similar issues for <sys/sbuf.h> usage during buildworld.

The problem details:

/head/sys/kern/subr_sbuf.c -r280193 added:

> #define SBUF_NULINCLUDED(s)     ((s)->s_flags & SBUF_INCLUDENUL)

and head (20150325 r280598) contains it.

But the SBUF_INCLUDENUL reference blocks buildworld (for at least /usr/local/bin/powerpc64-portbld-freebsd11.0-gcc (powerpc64-xtoolchain=gcc) use):

> /usr/local/bin/powerpc64-portbld-freebsd11.0-gcc -fpic -DPIC  -O2 -pipe   -std=gnu99 -fstack-protector -Wsystem-headers -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-pro
> totypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition
> -Wno-pointer-sign   -c /usr/src/lib/libsbuf/../../sys/kern/subr_sbuf.c -o subr_sbuf.So
> ...
> /usr/src/lib/libsbuf/../../sys/kern/subr_sbuf.c:73:45: error: 'SBUF_INCLUDENUL' undeclared (first use in this function)
> #define SBUF_NULINCLUDED(s) ((s)->s_flags & SBUF_INCLUDENUL)
>                                             ^

Looking to see where SBUF_INCLUDENUL usage and definitions might be in /usr/src for -r280598 ...

> # pwd
> /usr/src
> # find . \( -type d -name .svn -prune \) -or \( -type f -exec grep SBUF_INCLUDENUL {} \; -print \) | more
> .It Dv SBUF_INCLUDENUL
> ./share/man/man9/sbuf.9
> #define SBUF_INCLUDENUL 0x00000002      /* nulterm byte is counted in len */
> ./sys/sys/sbuf.h
>        sbuf_clear_flags(&sbuf, SBUF_INCLUDENUL);
> ./sys/vm/uma_core.c
>                SBUF_INCLUDENUL);
> ./sys/netinet/tcp_hostcache.c
>        sbuf_clear_flags(&sbuf, SBUF_INCLUDENUL);
> ./sys/kern/kern_malloc.c
>            SBUF_INCLUDENUL);
> ./sys/kern/kern_cons.c
>        sbuf_clear_flags(&sb, SBUF_INCLUDENUL);
>        sbuf_clear_flags(&sb, SBUF_INCLUDENUL);
> ./sys/kern/kern_descrip.c
>        sbuf_clear_flags(&sb, SBUF_INCLUDENUL);
>                sbuf_clear_flags(&sb, SBUF_INCLUDENUL);
>        sbuf_clear_flags(&sb, SBUF_INCLUDENUL);
>        sbuf_clear_flags(&sb, SBUF_INCLUDENUL);
>        sbuf_clear_flags(&sb, SBUF_INCLUDENUL);
> ./sys/kern/kern_proc.c
>        sbuf_new(&sb, NULL, 256, SBUF_AUTOEXTEND | SBUF_INCLUDENUL);
> ./sys/kern/kern_et.c
>        sbuf_new(&sb, NULL, 128, SBUF_AUTOEXTEND | SBUF_INCLUDENUL);
> ./sys/kern/kern_fail.c
>        s = sbuf_new(s, buf, length, SBUF_FIXEDLEN | SBUF_INCLUDENUL);
> ./sys/kern/kern_sysctl.c
> #define SBUF_NULINCLUDED(s)     ((s)->s_flags & SBUF_INCLUDENUL)
> ./sys/kern/subr_sbuf.c

Looking at the list of includes in /head/lib/libc/net/sctp_sys_calls.c for -r280193 shows:

> #include <sys/cdefs.h>
> __FBSDID("$FreeBSD: head/sys/kern/subr_sbuf.c 280193 2015-03-17 21:00:31Z ian $");
> 
> #include <sys/param.h>
> 
> #ifdef _KERNEL
> #include <sys/ctype.h>
> #include <sys/errno.h>
> #include <sys/kernel.h>
> #include <sys/malloc.h>
> #include <sys/systm.h>
> #include <sys/uio.h>
> #include <machine/stdarg.h>
> #else /* _KERNEL */
> #include <ctype.h>
> #include <errno.h>
> #include <stdarg.h>
> #include <stdio.h>
> #include <stdlib.h>
> #include <string.h>
> #endif /* _KERNEL */
> 
> #include <sys/sbuf.h>

That there was no complaint about sbuf.h being missing suggests that a <sys/sctp.h> was found but did not contain a SBUF_INCLUDENUL definition: so a different one than the above find/grep reported.

Using a find to report other sctp.h files shows:

> # find /usr/include \( -type d -name .svn -prune \) -or \( -type f -name sbuf.h -print \) | more
> /usr/include/sys/sbuf.h

The diff of the two variants shows the problem if the wrong file is found and used:

> # diff /usr/include/sys/sbuf.h /usr/src/sys/sys/sbuf.h
> 28c28
> <  *      $FreeBSD: head/sys/sys/sbuf.h 269179 2014-07-28 07:20:22Z gahr $
> ---
>> *      $FreeBSD: head/sys/sys/sbuf.h 279992 2015-03-14 16:02:11Z ian $
> 50a51
>> #define	SBUF_INCLUDENUL	0x00000002	/* nulterm byte is counted in len */
> 66a68,70
>> int		 sbuf_get_flags(struct sbuf *);
>> void		 sbuf_clear_flags(struct sbuf *, int);
>> void		 sbuf_set_flags(struct sbuf *, int);

Context details:

> make -j 8 CROSS_TOOLCHAIN=powerpc64-gcc 
> WITHOUT_CLANG_BOOTSTRAP= WITHOUT_CLANG= WITHOUT_CLANG_IS_CC= \
> WITHOUT_LLDB= \
> WITH_GCC_BOOTSTRAP= WITH_GCC= WITHOUT_GNUCXX= \
> WITHOUT_BOOT= WITHOUT_LIB32= \
> buildworld buildkernel \
> KERNCONF=GENERIC64vtsc-NODEBUG
> TARGET=powerpc TARGET_ARCH=powerpc64

> # svnlite info /usr/src
> Path: .
> Working Copy Root Path: /usr/src
> URL: https://svn0.us-west.freebsd.org/base/head
> Relative URL: ^/head
> Repository Root: https://svn0.us-west.freebsd.org/base
> Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f
> Revision: 280615
> Node Kind: directory
> Schedule: normal
> Last Changed Author: hselasky
> Last Changed Rev: 280598
> Last Changed Date: 2015-03-25 06:32:27 -0700 (Wed, 25 Mar 2015)

signals.h and pthread.h have been updated to more recent than -r280598 in order to avoid the __nonnull issues that exist as of -r280598.

> # svnlite st /usr/src --no-ignore
> ?       /usr/src/.snap
> ?       /usr/src/restoresymtable
> M       /usr/src/sys/ddb/db_main.c
> M       /usr/src/sys/ddb/db_script.c
> ?       /usr/src/sys/powerpc/conf/GENERIC64vtsc
> ?       /usr/src/sys/powerpc/conf/GENERIC64vtsc-NODEBUG
> ?       /usr/src/sys/powerpc/conf/GENERICvtsc
> ?       /usr/src/sys/powerpc/conf/GENERICvtsc-NODEBUG
> M       /usr/src/sys/powerpc/ofw/ofw_machdep.c
> M       /usr/src/sys/powerpc/ofw/ofwcall64.S

(The .c/.S changes are tied to a PowerMac-G5-specific boot-problem fix and getting information from early boot failures if I get any more. The GENERIC<?>'s remove ps3 in order to have both vt and sc at the same time.)

> # more /etc/src.conf 
> NO_WERROR=
> WITH_LIBCPLUSPLUS=
> CC=/usr/local/bin/powerpc64-portbld-freebsd11.0-gcc
> CXX=/usr/local/bin/powerpc64-portbld-freebsd11.0-g++
> CPP=/usr/local/bin/powerpc64-portbld-freebsd11.0-cpp
> CROSS_BINUTILS_PREFIX=/usr/local/powerpc64-freebsd/bin/
> X_COMPILER_TYPE=gcc
> CXXFLAGS+=-I/usr/obj/usr/srcC/tmp/usr/include/c++/v1/. -std=gnu++11 -L/usr/obj/usr/srcC/lib/libc++/.
> CXXFLAGS+=-I/usr/include/c++/v1/. -std=gnu++11 -L/usr/lib/.

(The above and just below experiments with mostly(?) avoiding use of gcc 4.2.1, even for CC/CXX/CPP contexts.)

> # ls -FPal /usr/bin/g[+c]*
> lrwxr-xr-x  1 root  wheel     48 Mar 20 02:03 /usr/bin/g++@ -> /usr/local/bin/powerpc64-portbld-freebsd11.0-g++
> lrwxr-xr-x  1 root  wheel     48 Mar 19 04:20 /usr/bin/gcc@ -> /usr/local/bin/powerpc64-portbld-freebsd11.0-gcc

> # more /etc/make.conf
> WRKDIRPREFIX=/usr/obj/portswork
> #WITH_DEBUG=
> MALLOC_PRODUCTION=

===
Mark Millard
markmi at dsl-only.net