svn commit: r292153 - head/lib/libc/regex/grot

[Bruce Evans]

On Sun, 13 Dec 2015, Garrett Cooper wrote:

> Log:
>  Add -static to CFLAGS to unbreak the tests by using a libc.a with
>  the xlocale private symbols exposed which aren't exposed publicly
>  via the DSO

This is an interesting hack.

I think there are some bad bugs in static libraries from exposing
public private (sic) symbols in them, and from using private public
(sic) symbols.  Or perhaps the reverse.

A public private symbol is one that is public (extern) in C but not
exported from the shared library.  A private public symbol is one
that is public (extern) in C and is also exported by the shared 
library, but is a weak symbol so it is sort of private for both.

An example of the latter is 'open' or syslog.  Both are in the
user namespace for Standard C.  syslog is in the user namespace
for some versions of POSIX.  libc is supposed to use renamed
versions so that it never uses a user version.

It mostly does this for _open.  open is a weak symbol so that
sloppy parts of libc and POSIX applications can see it.  Non-POSIX
applications can see it too unless they replace it.  According to
nm on libc.a, the only thing in the library with a sloppy reference
to open is citrus_mmap.o.  It also references close, fstat, mmap
and munmap.  This is probably OK since it is an extension of POSIX.

There is the following thicket of complications in names for what
should be the simple open syscall:

open.o:
X                  U __libc_interposing
X                  U __stack_chk_fail
X                  U __stack_chk_guard
X 0000000000000000 W open

_open.o:
X                  U .cerror
X 0000000000000000 T __sys_open
X 0000000000000000 W _open

Even _open is a weak symbol.  That makes no sense.  Similarly for all
syscalls except ones like _exit whose primary syscall name has the
single underscore.  I think it is just a bug in the PSEUDO macro in SYS.h.

Functions like syslog() are not handled so carefully.  They are never
renamed.  This was not a problem, since they were not called much from
other parts of libc.  Perhaps never for syslog(), or just from associated
parts with the closure of the parts being entirely inside or entirely
outside APIs that have syslog().  But this was broken by the stack
protector code.  The stack protector code is not called from all over
the library, e.g., for open as shown above.  This turns the careful
renaming for open into just an obfuscation (except the obfuscations also
give pessimizations).

Test program for this.

X #include <fcntl.h>
X #include <stdio.h>
X 
X #ifndef NO_DEBLOAT
X void
X openlog(void)
X {
X 	puts("opensyslog() is not in the Standard C library()");
X }
X 
X void
X syslog(void)
X {
X 	puts("syslog() is not in the Standard C library()");
X 	/* I am careful not to call this, but stack_protector.c isn't. */
X 	// system("rm -rf /");
X }
X 
X void
X vsyslog(void)
X {
X 	puts("vsyslog() is not even in POSIX");
X }
X 
X void
X closelog(void)
X {
X 	puts("opensyslog() is not in the Standard C library()");
X }
X 
X off_t
X lseek(int fd, off_t offset, int whence)
X {
X 	puts("lseek() is not in the Standard C library()");
X 	return -1;
X }
X #endif
X 
X int
X main(void)
X {
X #ifdef FORCE_USE
X 	openlog();
X 	vsyslog();
X 	closelog();
X #else
X 	open("/dev/null", 0);	/* warm up for debugging */
X 	open("/dev/null", 0);	/* try to get it to call us */
X #endif
X 	puts("hello world is bloated");
X }

This must be run under gdb.  Manually corrupt the stack so that
__stack_chk_fail is called.  Then the private syslog() is called iff
the linkage is static.  The other functions are normally not called
since __stack_chk_fail kills the program with SIGABRT after calling
syslog().

Debugging this shows another bug (bogusness at least): _open uses the
open syscall, but open uses the openat syscall.

Debugging this is especially difficult when it is dynamically linked.
Then the __libc_interposing, __stack_chk_fail and __stack_chk_guard
symbols are so private that even the debugger can't see them, at least
with the library not compiled with -g.  With static linkage, they are
normal public symbols.

This program was originally for testing reduction of library bloat.
The bloat is so large that the null program main(){} now links to
syslog and might even call it if the stack gets corrupted.  So the
null program now has size more than 460KB on amd64 (more than 4
times larger than /bin/sh in FreeBSD-1).

Bruce