Observations on virtual memory operations

[Michael Schuster]

On Tue, Dec 29, 2020, 00:37 Pete Wright <pete at nomadlogic.org> wrote:

>
>
> On 12/28/20 3:25 PM, doug wrote:
> > I have two servers running jails that "routinely" run out of swapspace
> > with
> > no demand paging activity. To try and get a handle on VM/swapspace
> > management I have been tracking swapinfo vs memory use as measured by
> > top.
> > The numbers do not exactly add up but I assume that is not involved in my
> > issue.
> >
> <snip>
> >
> > The other day I caught the system at 73% swapspace used. At this level
> > the
> > system was in a near thrashing state in that typing a key got it
> > echoed in
> > 10 <--> 30 seconds. There was about 600MB of swapspace at this point. I
> > would think there is no way to debug this except as a thought experiment.
>
> The first thing that comes to mind is do you have the ability to hook
> any metrics/monitoring onto this system.  For example, I use collectd on
> my systems to report overall CPU/memory metrics as well as per-process
> memory metrics.
>
> Alternatively you could write a simple shell script that run's "ps" and
> parses the output of memory utilization on a per-process basis.
>
> either of the above approaches should give you some insight into where
> the memory leak is coming from (assuming you already do not know).
>
> one trick i use is to invoke a process with "limits" to ensure it does
> not exceed a certain amount of memory that I allocate to it. for example
> with firefox i do this:
> $ limits -m 6g -v 6g /usr/local/bin/firefox
>
> that should at least buy you enough time to investigate why the process
> needs so much memory and see what you can do about it.
>

If the usual observation tools (and please note, I don't have too much
specific knowledge here) don't tell you what you need to know, have a look
at DTrace. Again, I have no (active) specific knowledge; personally, I
would start with Brendan Gregg's tools (ask your favourite search
engine)...

Regards
Michael