resident memory limit

Brad Penoff penoff at cs.ubc.ca
Sat May 31 02:07:07 UTC 2008 

On Wed, May 21, 2008 at 11:46 AM, Joshua Isom <jrisom at gmail.com> wrote:
>
> On May 21, 2008, at 12:36 PM, Bill Moran wrote:
>
>> In response to "Brad Penoff" <penoff at cs.ubc.ca>:
>>>
>>> On Wed, May 21, 2008 at 6:09 AM, Bill Moran <wmoran at potentialtech.com>
>>> wrote:
>>>>
>>>> In response to "Brad Penoff" <penoff at cs.ubc.ca>:
>>>>
>>>>> On Tue, May 20, 2008 at 2:39 PM, Brad Penoff <penoff at cs.ubc.ca> wrote:
>>>>>>
>>>>>> On Tue, May 20, 2008 at 1:54 PM, Bill Moran <wmoran at potentialtech.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> In response to "Brad Penoff" <penoff at cs.ubc.ca>:
>>>>>>>>
>>>>>>>> I have an application that runs on Linux or Mac OS X but seems to
>>>>>>>> have
>>>>>>>> a problem when I run on FreeBSD (6.3 or 7).  The issue is the memory
>>>>>>>> footprint for the application (osubw_sctpclien below) is quite
>>>>>>>> large;
>>>>>>>> on Linux it can be as much as 950 MB in resident memory, according
>>>>>>>> to
>>>>>>>> top.  However, on FreeBSD I start to get ENOMEM always around the
>>>>>>>> time
>>>>>>>> my resident memory size is about 200 MB.
>>>>>>>>
>>>>>>>> I read a few posts and have seen people fixing their problems by
>>>>>>>> adjusting kern.maxdsiz in /boot/loader.conf and/or by adding a swap
>>>>>>>> file.  I've tried both and for my application, it still seems to be
>>>>>>>> limited to 200 MB resident memory regardless of maxdsize and swap
>>>>>>>> file
>>>>>>>> setting.  I wrote a toy application (malloctest below) that calls
>>>>>>>> malloc in a while(1) and breaks once it gets ENOMEM (doing another
>>>>>>>> while(1) so it doesn't exit); this application's memory size in top
>>>>>>>> always matches the kern.maxdsiz setting, however it has a very low
>>>>>>>> resident memory number, according to top.
>>>>>>>
>>>>>>> Have a look at /etc/login.conf and the associated man pages.
>>>>>>>
>>>>>
>>>>> BTW, we've seen the exact behavior on FreeBSD 7 as well (6.3 was
>>>>> reported here).  We've tried on different hardware as well, and keep
>>>>> getting haunted by this resident memory limit that we don't know how
>>>>> to set.
>>>>>
>>>>> Any idea why, in the data I originally reported, I can allocate
>>>>> kern.maxdsiz + swap (see SIZE from top output) for malloc(1 MB) in a
>>>>> while loop, yet the top value for RES is always really low?
>>>>>
>>>>> How come, in contrast, my application starts to report ENOMEM when
>>>>> SIZE is 203 MB and RES is 201 MB?  This is why I titled the thread
>>>>> asking about an unknown (to me ;-) limit for resident memory...
>>>>
>>>> It's called memory overcommit.  If the OS thinks it _might_ be able
>>>> to get you the memory, it will allow it.  You only actually use the
>>>> memory when you start putting data in it (hence the difference between
>>>> SIZE and RES)  Add a statement to fill up the malloc()ed memory with
>>>> some sort of data in your loop, and you'll see different behaviour.
>>>>
>>>> As to what's limiting your application, I'm not sure.  What does the
>>>> output of 'ulimit -a' say?
>>>>
>>>
>>> Thanks again for your time.
>>>
>>>
>>> With the default loader.conf, my "limit -a" output is:
>>>
>>> Resource limits (current):
>>>  cputime          infinity secs
>>>  filesize         infinity kB
>>>  datasize           524288 kB
>>>  stacksize           65536 kB
>>>  coredumpsize     infinity kB
>>>  memoryuse        infinity kB
>>>  memorylocked     infinity kB
>>>  maxprocesses         5547
>>>  openfiles           11095
>>>  sbsize           infinity bytes
>>>  vmemoryuse       infinity kB
>>>
>>> My application starts getting ENOMEM when I have 201 MB of resident
>>> memory.
>>>
>>>
>>>
>>> When I change my loader.conf to match the 2 GB of physical memory that I
>>> have:
>>> kern.maxdsiz="2147483648"
>>> kern.maxssiz="2147483648"
>>> kern.dfldsiz="2147483648"
>>>
>>> ...and reboot, then my "limit -a" output is:
>>>
>>> Resource limits (current):
>>>  cputime          infinity secs
>>>  filesize         infinity kB
>>>  datasize          2097152 kB
>>>  stacksize         2097152 kB
>>>  coredumpsize     infinity kB
>>>  memoryuse        infinity kB
>>>  memorylocked     infinity kB
>>>  maxprocesses         5547
>>>  openfiles           11095
>>>  sbsize           infinity bytes
>>>  vmemoryuse       infinity kB
>>>
>>>
>>> However, the application still seems to max out at 201 MB of resident
>>> memory.
>>>
>>>
>>> People suggest to fix my login.conf but the memory related fields are
>>> set to unlimited... Any ideas where this 201 MB limit of resident
>>> memory comes from?
>>
>> That's pretty strange.  If I had to guess, I would guess that there is no
>> 201M limit, but that you're hitting some other limit that just happens to
>> predictably occur at 201M with that program.
>>
>> I'm kind of grasping at straws here, so hopefully I won't lead you on a
>> wild goose chase, but I would look next at putting some debugging in
>> /etc/malloc.conf and seeing if you get any useful information from it
>> (see the malloc man page).  From there, possibly a ktrace of the process.
>> Hopefully you have source code for the program and can compile it with
>> debugging and run it under gdb.
>>
>> --
>> Bill Moran
>> http://www.potentialtech.com
>> _______________________________________________
>> freebsd-questions at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
>> To unsubscribe, send any mail to
>> "freebsd-questions-unsubscribe at freebsd.org"
>>
>>

I'm still playing around with malloc.conf and ktrace, searching for
answers (I'll report if I find any) but in the meantime, I'll try to
answer your questions...

>
> Here's a question I haven't seen asked yet.  How much memory is it trying to allocate?  If it can't get everything it's asking for it can fail.

This same application runs on both Mac OS X and Linux.  On Linux, the
maximum memory demands shown in "top" were VIRT=955MB RES=929MB.  This
was actually on a machine with only 1 GB of RAM.  This same program
starts getting ENOMEM on FreeBSD on a machine with 2 GB of RAM when
RES is 201 MB; the same occurs on FreeBSD on a machine with 1 GB of
RAM (same HW as Linux).  On FreeBSD, I can run a dummy program that
calls malloc in a loop essential until the value of SIZE on "top"
(same as VIRT in Linux) is kern.maxdsiz (which I set to 2GB in
loader.conf).

> Also, how is the application being started?  There could be some setting in the shell startup that's putting a limit.

I run it as root which uses /bin/csh .  I did a chsh as root to bash
and retried the same app but still reached the same magic 201 MB
limit.

> Is it a native FreeBSD program or is it a linux executable running under simulation?

It's a native FreeBSD program.


brad

More information about the freebsd-questions mailing list