two 4GB mallocs => SEGV

[Peter Wemm]

On Tuesday 26 October 2004 02:32 pm, David O'Brien wrote:
> On Tue, Oct 26, 2004 at 12:55:06PM -0500, James R. Van Artsalen wrote:
> > David O'Brien wrote:
> > >malloc.c:map_pages() calls brk(2) and this is where it goes to
> > > la-la land.
> >
> > The brk() kernel call is probably failing due to ulimit being
> > exceeded and not anything mysterious.
>
>     # ulimit -a | grep virt
>     virtual memory        (kbytes, -v) unlimited

The data size limit is 8GB though.

> BTW, a statically built binary (ie, using libc.a) just hangs in the
> brk(2) call.

Yes, and that is rather worrying.

> > A few months ago I posted this bug in the libc brk(2) code - the
> > stack is not balanced if the kernel returns an error.  I'm not
> > running current code at the moment but see if you brk.S has a stack
> > issue at the err: label.  Stick in this pop if so and report if
> > malloc(3c) then returns NULL instead of crashing, then up your
> > ulimit and try again and see if all works without error.
> >
> > --- lib/libc/amd64/sys/brk.S.~1~        Sat May 24 12:35:23 2003
> > +++ lib/libc/amd64/sys/brk.S    Fri Apr  9 02:02:22 2004
> > @@ -78,6 +78,7 @@
> >        popq    %rdi
> >        ret
> > err:
> > +       popq    %rdi
> > #ifdef PIC
> >        movq    PIC_GOT(HIDENAME(cerror)),%rdx
> >        jmp     *%rdx
>
> Today's code has:
>
>     err:
>             addq    $8, %rsp
>     #ifdef PIC
>             movq    PIC_GOT(HIDENAME(cerror)),%rdx
>             jmp     *%rdx
>
>
> so the stack should be OK.

Where do you see the addq?  Its not in cvs that I can see..  Oh, its in 
sbrk.S - this patch is against brk.S.

-- 
Peter Wemm - peter at wemm.org; peter at FreeBSD.org; peter at yahoo-inc.com
"All of this is for nothing if we don't go to the stars" - JMS/B5