FreeBSD using swap even though there's a lot of free memory

[Doug Hardie]

> On 16 October 2020, at 11:55, Steve O'Hara-Smith <steve at sohara.org> wrote:
> 
> On Fri, 16 Oct 2020 11:35:42 -0700
> Michael Sierchio <kudzu at tenebras.com> wrote:
> 
>> On Thu, Oct 15, 2020 at 7:24 AM Twingly Customer Support
>> <team at twingly.com> wrote:
>> 
>>> Hi,
>>> 
>>> We have a server running FreeBSD 12.1-RELEASE-p10. We currently have a
>>> problem where FreeBSD starting to swap when running ZFS scrub, even
>>> though we have ~70G of free memory.
>> 
>> 
>> tl;dr – this is normal, nothing to see, move along.
> 
> 	Not when it starts running out of swap space while still reporting
> 70G of free memory.

I encountered this issue a year or so ago.  In my case it turned out to be a process that was allocating anonymous segments using mmap.  It tended to forget to free them after the usage was complete.  As a result, the system would run out of swap even though there was plenty of free memory.  Anonymous segments are mapped to swap space.  I traced the problem using:

	procstat -va | grep df

That generated a lot of data.  I looked for processes that had an increasing number of df files.  Eventually I found the right one and fixed it.

Interesting observation:  I have used FreeBSD since version 2.5 and for years it always had a small number of processes swapped out within an hour of a boot.  Typically they were gettys and similar processes that never got used.  However, with 12.x, I now find that swap info is always showing 0 blocks used even after weeks of operation.  Something has changed for the better.

It is also possible that greping for sw rather than df might give some useful information.  I can't tell as none of my systems are using any swap.  Actually, one system did not have any swap for over a year.  I didn't notice that there was an issue mounting its swap.  There were no performance indicators that issue existed.  Nothing changed after I corrected the problem and added the swap.

-- Doug