lightly loaded system eats swap space

[Adam]

On Sun, Jun 17, 2018 at 5:19 PM, tech-lists <tech-lists at zyxst.net> wrote:

> Hello list,
>
> context is (server)
> freebsd-11-stable r333874, ZFS raidz1-0 (3x4TB disks), 128GB RAM, E5-2630
> @2.3GHz, generic kernel.
>
> There's one bhyve guest on this server (using 4x cpu and 16GB RAM, also
> freebsd-11-stable)
>
> There have been no special options for zfs configuration on the server,
> apart from several datasets having the compressed property set (lz4).
>
> The server runs nothing else really apart from sshd and it uses ntpd to
> sync local time.
>
> How come such a lightly loaded server with plenty of resources is eating
> up swap? If I run two bhyve instances, i.e. two of the same size as
> indicated above, so 32GB used for the bhyves, I'll get out-of-swapspace
> errors in the daily logs:
>
> +swap_pager_getswapspace(24): failed
> +swap_pager_getswapspace(24): failed
> +swap_pager_getswapspace(24): failed
>
> Here's top, with one bhyve instance running:
>
> last pid: 49494;  load averages:  0.12,  0.13,  0.88
>
>                              up 29+11:36:06  22:52:45
> 54 processes:  1 running, 53 sleeping
> CPU:  0.4% user,  0.0% nice,  0.4% system,  0.3% interrupt, 98.9% idle
> Mem: 8664K Active, 52M Inact, 4797M Laundry, 116G Wired, 1391M Buf, 4123M
> Free
> ARC: 108G Total, 1653M MFU, 105G MRU, 32K Anon, 382M Header, 632M Other
>      103G Compressed, 104G Uncompressed, 1.00:1 Ratio
> Swap: 4096M Total, 3502M Used, 594M Free, 85% Inuse
>
>   PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU
> COMMAND
> 49491 root          1   4    0 16444K 12024K select  9   0:12   6.49% ssh
> 32868 root         12  20    0  9241M  4038M kqread  2  23.2H   1.30% bhyve
> 49490 root          1  20    0 10812K  6192K sbwait  5   0:02   0.88% sftp
>
> From the looks of it, a huge amount of ram is wired. Why is that, and how
> would I debug it?
>

That seems to be shown in the output you provided:
ARC: 108G Total, 1653M MFU, 105G MRU, 32K Anon, 382M Header, 632M Other


>
> A server of similar spec which is running freebsd-current with seven bhyve
> instances doesn't have this issue:
>

Based upon the output neither ram nor swap seems like similar spec so I
wonder if you could say what you mean by that.

-- 
Adam