poudriere, swap full and top says memory is free ?

[Trev]

Don Lewis wrote on 15/09/2019 11:31:
> On 14 Sep, John-Mark Gurney wrote:
>> Kurt Jaeger wrote this message on Sat, Sep 14, 2019 at 19:38 +0200:
>>> - a poudriere build
>>> - of a list of ports
>>> - on 12.0-RELEASE-p10
>>> - on a 4 core+4 hyperthreads CPU, an Intel(R) Xeon(R) CPU E3-1230 v6
>>>    @ 3.50GHz
>>> - with 32 GB RAM
>>> - zpool with 2x 500 GB SSDs as a mirror
>>>
>>> and right now, this can be seen:
>>>
>>> last pid: 90922;  load averages:  5.02,  5.14,  5.73    up 0+03:53:08  19:31:05
>>> 82 processes:  6 running, 76 sleeping
>>> CPU: 60.6% user,  0.0% nice,  2.1% system,  0.0% interrupt, 37.3% idle
>>> Mem: 4598M Active, 2854M Inact, 11G Laundry, 6409M Wired, 6375M Free
>>> ARC: 3850M Total, 1721M MFU, 2090M MRU, 665K Anon, 19M Header, 19M Other
>>>       3406M Compressed, 3942M Uncompressed, 1.16:1 Ratio
>>> Swap: 18G Total, 18G Used, 396K Free, 99% Inuse, 68K In
>>>
>>> So: Swap is full, approx. 6 GB memory is reported as free.
>>>
>>> This is surprising. Can I somehow tune this in any way, so that
>>> the memory available is used for the build ? Or is the problem somewhere
>>> else ?
>>
>> Are you sure that this hasn't just recently completed a large link of
>> something like Chromium?  There are known to be compiles that can take
>> many GB's of memory and if they recently exited, there hasn't been time
>> to swap stuff back in...  or is this the steady state over the entire
>> compile?
> 
> This is sort of an odd case.  I suspect that swap filled and then a
> process that was using a large amount of memory but no swap exited or
> was killed.  That freed a bunch of memory, but no swap.
> 
> I'm pretty sure that when a memory page is paged back in from swap, that
> the copy in swap is retained and not deallocated.  Under memory
> pressure, that allowed the page to be stolen without having to write it
> back out to swap again, unless it was re-dirtied in the meantime.

Don't forget swap fragmentation could conceivably cause oom even if 
there is swap appearing to be available. sysctl vm.swap_fragmentation is 
interesting.