AVILA getting close!

[Andrew Turner]

On Sat, 21 Jun 2014 22:16:44 -0700
John-Mark Gurney <jmg at funkthat.com> wrote:

> Andrew Turner wrote this message on Sat, Jun 21, 2014 at 17:01 +0100:
> > On Sat, 21 Jun 2014 00:08:28 -0700
> > John-Mark Gurney <jmg at funkthat.com> wrote:
> > > I've looked at the assembly for donelist_init for both the working
> > > and non-working case, and besides an extra store in the good
> > > version, which I think is just because of the unoptimized code,
> > > things look the same to me...
> > I suspect it's more likely the working version is not making
> > assumptions on the stack alignment, where the broken one is.
> 
> So, is there a way you can fix up the rtld tramp in
> rtld-elf/arm/rtld_start.S so that the stack can be traced through?
> 
> Right now, when I'm in the plt code, I can get the userland stack,
> but once I end up in _rtld_bind_start, I can't get a trace back
> past the got which is making this next to imposible to debug..
> 
> (gdb) stepi
> 0x200774d4 in .plt () from /lib/libc.so.7
> 2: $pc = 0x200774d4
> 1: $sp = 0xbffff24c
> (gdb) bt
> #0  0x200774d4 in .plt () from /lib/libc.so.7
> #1  0x20171804 in getenv (name=0x20184a98 "MALLOC_CONF")
>     at /usr/src.avila/lib/libc/stdlib/getenv.c:144
> #2  0x200f23b0 in malloc_init_hard () at jemalloc_jemalloc.c:464
> #3  0x200f34d8 in jemalloc_constructor () at jemalloc_jemalloc.c:296
> #4  0x2001dee0 in objlist_call_init (list=<value optimized out>, 
>     lockstate=0xbffffb40)
> at /usr/src.avila/libexec/rtld-elf/rtld.c:2385 #5  0x200212b8 in $a
> () at /usr/src.avila/libexec/rtld-elf/rtld.c:640 #6  0x200212b8 in $a
> () at /usr/src.avila/libexec/rtld-elf/rtld.c:640 (gdb) stepi
> _rtld_bind_start ()
> at /usr/src.avila/libexec/rtld-elf/arm/rtld_start.S:80
> 80              stmdb   sp!,{r0-r4,sl,fp} 2: $pc = 0x20018f84
> 1: $sp = 0xbffff24c
> Current language:  auto; currently asm
> (gdb) bt
> #0  _rtld_bind_start ()
> at /usr/src.avila/libexec/rtld-elf/arm/rtld_start.S:80 #1  0x2019e0c0
> in .got () from /lib/libc.so.7 #2  0x2019e0c0 in .got ()
> from /lib/libc.so.7
> 
> Poof, now I have no clue where I am anymore...  :(  Or at least
> provide an option (enabled by -DDEBUG) to provide it..
> 
> Well, I did manage to work this out.. and I feel a little stupid since
> I saw this name and thought it a bit weird..  I believe that gcc is
> is not making sure that the stack is aligned when calling
> __aeabi_read_tp...  I believe that gcc does not think that it is a
> full function, but it does go through relocation... If you look at the
> disassembly of the sob function in newsyslog, you'll see that it will
> leave the stack unaligned off the bat:
>     9f58:       e92d4030        push    {r4, r5, lr}
> 
> and setting a breakpoint there confirms this:
> (gdb) c
> Continuing.
> 
> Breakpoint 8, sob (
>     p=0xbfffef20 "/var/log/all.log\t\t\t600  7\t   *\t at T00  J\n")
>     at /usr/src.avila/usr.sbin/newsyslog/newsyslog.c:2400
> 2400            while (p && *p && isspace(*p))
> 2: $pc = 0x9f5c
> 1: $sp = 0xbfffeeec
> 
> So, gcc is generating broken code...  Can you please fix this?
> 
> Thanks.
> 

We tracked this down to a problem where, normally the stack is aligned
to an 8-byte boundary, however in leaf functions the stack may be
aligned to a 4-byte boundary. The problem is gcc and upstream
clang/llvm don't detect that functions that use thread local storage as
non-leaf. Functions that use tls call __aeabi_read_tp(). As this is
implemented in libc it requires the symbol to be resolved first, however
as the compiler doesn't see this as a function call it may align the
stack to a non-8-byte address.

I have a fix for this that has been tested on armeb. I'll commit it
soon, after I've tested it on other platforms.

Andrew