Re: Swap filling up, usermode process swap usage doesn't explain

From: Scott Gasch <scott.gasch_at_gmail.com>
Date: Wed, 19 Jul 2023 22:11:30 UTC
Yes, I'm using ZFS.  Here's what top says:

last pid: 88926;  load averages:  1.20,  0.96,  0.87
 up 5+17:48:34  15:09:58
274 processes: 1 running, 272 sleeping, 1 zombie
CPU:  1.8% user,  0.0% nice,  0.5% system,  0.0% interrupt, 97.8% idle
Mem: 1844M Active, 7777M Inact, 77G Laundry, 35G Wired, 750M Buf, 3367M Free
ARC: 24G Total, 2878M MFU, 18G MRU, 21M Anon, 119M Header, 2622M Other
     18G Compressed, 25G Uncompressed, 1.33:1 Ratio
Swap: 144G Total, 11G Used, 133G Free, 7% Inuse

If I leave this alone it will grow to consume all available swap space.
I'll try your fix with the sysctl knob and see what happens...  I hope this
is it, I've been fighting this for a while now.

Thx,
Scott


On Wed, Jul 19, 2023 at 11:34 AM Pete Wright <pete@nomadlogic.org> wrote:

>
>
> On 7/19/23 07:49, Scott Gasch wrote:
> > I am running a 13.2-RELEASE GENERIC kernel and seeing a pattern where,
> > after about 10 days of uptime, my swap begins to fill up.
> >
> <snip>
> >
> > At least they agree about it being 11G.  Is this kernel memory being
> > paged out to swap?  The machine has 128G of physical memory and isn't
> > under very heavy load at the moment.
> >
>
> Are you running ZFS by any chance?  If so its possible it is trying to
> use as much memory as possible for the ARC.  I've seen this on a few
> systems which lots of memory.  One way to tell is to run "top" and look
> at the ARC stats:
>
> last pid: 71322;  load averages:  1.02,  0.94,  0.87              up
> 8+18:38:34  11:31:26
> 376 processes: 1 running, 146 sleeping, 229 zombie
> CPU:  0.6% user,  0.0% nice,  6.5% system,  0.0% interrupt, 93.0% idle
> Mem: 3599M Active, 18G Inact, 4132M Laundry, 4272M Wired, 892M Free
> ARC: 1749M Total, 651M MFU, 239M MRU, 1864K Anon, 13M Header, 844M Other
>       216M Compressed, 758M Uncompressed, 3.52:1 Ratio
>
>
>
> On a few of my larger memory systems I cap the ARC by setting this
> sysctl knob (this is like 45G on my system):
> vfs.zfs.arc.max=45000000000
>
>
> -pete
>
>
> --
> Pete Wright
> pete@nomadlogic.org
> @nomadlogicLA
>