two 4GB mallocs => SEGV

[David O'Brien]

On Tue, Oct 26, 2004 at 12:55:06PM -0500, James R. Van Artsalen wrote:
> David O'Brien wrote:
> 
> >malloc.c:map_pages() calls brk(2) and this is where it goes to la-la land.
> >
> > 
> >
> The brk() kernel call is probably failing due to ulimit being exceeded 
> and not anything mysterious.

    # ulimit -a | grep virt
    virtual memory        (kbytes, -v) unlimited

BTW, a statically built binary (ie, using libc.a) just hangs in the
brk(2) call.

> A few months ago I posted this bug in the libc brk(2) code - the stack 
> is not balanced if the kernel returns an error.  I'm not running current 
> code at the moment but see if you brk.S has a stack issue at the err: 
> label.  Stick in this pop if so and report if malloc(3c) then returns 
> NULL instead of crashing, then up your ulimit and try again and see if 
> all works without error.
> 
> --- lib/libc/amd64/sys/brk.S.~1~        Sat May 24 12:35:23 2003
> +++ lib/libc/amd64/sys/brk.S    Fri Apr  9 02:02:22 2004
> @@ -78,6 +78,7 @@
>        popq    %rdi
>        ret
> err:
> +       popq    %rdi
> #ifdef PIC
>        movq    PIC_GOT(HIDENAME(cerror)),%rdx
>        jmp     *%rdx

Today's code has:

    err:
            addq    $8, %rsp
    #ifdef PIC
            movq    PIC_GOT(HIDENAME(cerror)),%rdx
            jmp     *%rdx


so the stack should be OK.

-- 
-- David  (obrien at FreeBSD.org)