Re: swap/page problem

Hi,

On Thu, Feb 10, 2022 at 02:28:50PM -0800, Kevin Oberman wrote:
>During a large build (llvm13), my system ground to a near halt with almost
>everything suspended. After several minutes, the system slowly recovered.
>When I looked at the messages log, I found 57 kernel messages spread over
>3.75 hours, in the form of:
>wap_pager: indefinite wait buffer: bufobj: 0, blkno: 862845, size: 20480
>
>The block numbers and sizes varied. bufobj was always '0'. I had
>significant swap available swap space,as far as I could tell. I have 20GB
>of RAM and 24GB of swap. I am running stable 48937-3c6b6246f2f from January
>13.
>
>I know that the LLVM build is huge, but I've not seen this before. What,
>exactly, is this message telling me? Am I out of RAM and swap? I couldd add
>another 24GB of swap, though it would be on spinning rust, not SSD.

I've seen this problem before and came to the conclusion 
after reading threads on the lists and asking questions that 
it wasn't a swap problem as primary issue but maybe theres some contention 
between processes which affects llvm in particular and makes it eat swap.
Also, as you noted, swap doesn't really run out.

I use poudriere to build and have parallel jobs set to 1 now 
with make jobs enabled and the problem doesn't happen. If you're just
using the ports tree in the traditional way, try make -j1.
If parallel jobs is unset it would use hw.ncpu which here is 8 which would
produce the problem you descrive when compiling llvm13.

These sysctls are now set:

vfs.read_max=128                       # default 64 - speeds up disk i/o
vfs.aio.max_buf_aio=8192
vfs.aio.max_aio_queue_per_proc=65536
vfs.aio.max_aio_per_proc=8192
vfs.aio.max_aio_queue=65536
vm.pageout_oom_seq=120
vm.pfault_oom_attempts=-1

those last two may be especially helpful for your situation.

My context here is amd64 i7-4770K (so 8 cpus with HT) clocked to 4.3GHz
and the disk used for building poers is SSD wiht 16GB swap (partition).
RAM is 32GB

-- 
J.