Is maximum swap usage tunable?

[bob prohaska]

On Sun, Mar 04, 2018 at 10:30:17PM -0700, Warner Losh wrote:
> 
> I'd look at power. And add more swap.
> 

Both have now been done. Voltage at the GPIO header is 5.01-5.04, 
idle all the way up to make -j4 buildworld.

Swap was increased to 3 GB in two partitions on the microSD, with
the usb-flash partition turned off for now.

As make -j4 buildworld's logfile approached the 1.6 MB mark, I started
gstat -a -d -B -I 1s > j4_3Gswap2_gstat.log and began watching the top
display. Pretty soon, swap usage began to rise, peaking a little over
700 MB. Idle time remained low, no swread or pfault states were observed
in top. 

Here's a snippet of gstat output from about this time:

dT: 1.003s  w: 1.000s
 L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w    d/s   kBps   ms/d   %busy Name
    7    878    235   2213    9.9    643   3808    6.2      0      0    0.0  100.0  mmcsd0
    7    878    235   2213    9.9    643   3808    6.2      0      0    0.0  100.0  mmcsd0s3
    5    444    125   1228    9.6    319   1898    6.2      0      0    0.0  100.1  mmcsd0s3b
    2    435    111    985   10.3    324   1910    6.1      0      0    0.0   99.8  mmcsd0s3d

Perhaps an hour later the swap usage again rose, to somewhere around
500 MB, but swread and pfault began to appear in the top window's state
column, and the idle time approached 50%, occasionally exceeding it.
It's hard to locate the spot in the gstat output corresponding, but I 
think this is representative:

dT: 1.006s  w: 1.000s
 L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w    d/s   kBps   ms/d   %busy Name
    6    240     22    219   11.5    218   2795   13.2      0      0    0.0   88.9  mmcsd0
    6    240     22    219   11.5    218   2795   13.6      0      0    0.0   90.2  mmcsd0s3
    3    123     15    179   10.3    108   1380   16.3      0      0    0.0   91.4  mmcsd0s3b
    3    116      7     40   14.3    109   1416   11.9      0      0    0.0   68.7  mmcsd0s3d

The machine got over this hump too, and is still running as I write.
The log file is up to ~15MB, in the "building libraries" stage.

It's curious that the initial burst of swap use seems to be handled more
gracefully than the later, smaller, flurry of activity. Moreover, both are
far below the 3GB swap limit of the partitions active. There is no warning
from the kernel about too much swap space being allocated. Nor are there
any "indefinite wait..." messages on the console.

As an aside, the write speed of 2.8 MB/sec for the microSD card corresponds
fairly well with the claimed 4k random write speed of 2.2 MB/sec for SanDisk
Extreme flash memory. I'm not sure how this compares to a mechanical hard
disk, but it _used_ to be ok.

I note in passing that r330367 armv7  on a Pi2 also halts  a
make -j4 buildworld with 2GB of USB-hosted swap available. A previous 
revision, a couple of weeks older, ran to completion with the same 
configuration. 

Thanks for reading, if anybody can suggest more informative tests please do.
Buildworld is a very clumsy stress test, Peter Holm's stress2 didn't
compile the last time I tried it, many months ago.

bob prohaska