amd64/124134: The kernel doesn't follow the calling convention in the SVR4/i386 ABI

[Pedro F. Giffuni]

>Number:         124134
>Category:       amd64
>Synopsis:       The kernel doesn't follow the calling convention in the SVR4/i386 ABI
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-amd64
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Fri May 30 14:10:03 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator:     Pedro F. Giffuni
>Release:        6.3-Release
>Organization:
>Environment:
FreeBSD kakumen.cable.net.co 6.3-RELEASE FreeBSD 6.3-RELEASE #10: Sat Jan 19 01:13:55 COT 2008     root at kakumen.cable.net.co:/usr/src/sys/amd64/compile/SMP  amd64

>Description:
While porting glibc to FreeBSD it was found that FreeBSD doesn't strictly conform to the SVR4 ABI on AMD64:
http://www.x86-64.org/documentation/abi.pdf (see startup calling convention in 3.4.1.)

Further explanation from Petr Salinger:

/* This is the canonical entry point, usually the first thing in the text
    segment.  The SVR4/i386 ABI (pages 3-31, 3-32) says that when the entry
    point runs, most registers' values are unspecified, except for a few.
    Blindly applied on amd64:

    %rdx         Contains a function pointer to be registered with `atexit'.
                 This is how the dynamic linker arranges to have DT_FINI
                 functions called for shared libraries that have been loaded
                 before this code runs.

    %rsp         The stack contains the arguments and environment:
                 0(%rsp)                 argc
                 8(%rsp)                 argv[0]
                 ...
                 (8*argc)(%rsp)          NULL
                 (8*(argc+1))(%rsp)      envp[0]
                 ...
                                         NULL

    But on amd64 %rsp also have to be 16-byte aligned,
    standard C calling convention already passes arguments in registers.

    FreeBSD uses %edi as pointer to arguments and environment, %rsp is passed aligned.
    On entry from kernel, %rsp=%rdi or %rsp=%rdi-8,
    on entry from ld.so, glibc might set up it slightly differently.

    On FreeBSD, we use %rsi for passing function pointer to rtld_fini().
    On entry from FreeBSD kernel, %rsi is cleared, %rdx is not cleared,
    on entry from ld.so, glibc sets both %rsi and %rdx to point to rtld_fini().

    Used interface (via %rdi, %rsi) is equal to standard C calling interface for

    void _start(void *arg, void *rtld_fini());

*/
>How-To-Repeat:
http://lists.debian.org/debian-bsd/2006/02/msg00223.html
>Fix:
It can be workaround in FreeBSD kernel by following both standards simultaneously,
i.e. on entry from kernel after exec() should be %rsp=%rdi and %rdx=%rsi, and %rsp should be properly aligned.


>Release-Note:
>Audit-Trail:
>Unformatted: