Re: What can I learn about data that is staying paged out? (There is a more specific poudriere bulk related context given.)

From: Mark Millard <marklmi_at_yahoo.com>
Date: Sat, 11 Jun 2022 22:42:24 UTC
I got evidence finally. An example is the sequence:

# swapoff /dev/gpt/CA72USBswp14 /dev/gpt/CA72USBswp16
# swapon /dev/gpt/CA72USBswp14 /dev/gpt/CA72USBswp16

(Wait for a time letting some "staying paged out" space
accumulate.)

# du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/
319	/usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/

(Wait for a time letting some more "staying paged out" space
accumulate.)

# du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/
307	/usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/

(The size decreases seem to roughly track the sustained Swap
Used increases. [There is  some other paging activity but it
is not a lot.])

# swapoff /dev/gpt/CA72USBswp14 /dev/gpt/CA72USBswp16

# du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/
379	/usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/

(Apparently, du -xsm does not count the paged out
pages for tmpfs but the pages show up when forced
back into RAM.)

# swapon /dev/gpt/CA72USBswp14 /dev/gpt/CA72USBswp16

# du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/
379	/usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/

(I expect that when I see the swap usage grow in a staying
paged out manor, I'll find that this -xsm figure will also
have decreased on the retry in the new context.)


The primary space in question is actually in a subdirectory:

Taking a quick estimate of the file count in . . ./ref/.p/var/cache/
I get:

# ls -Tla /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ | wc
 82146  821452 7942485

So something like 82144 files ( ignoring . and .. ).

# ls -Tla /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ | more
total 301234
drwxr-xr-x  2 root  wheel  5257152 May 28 02:31:26 2022 .
drwxr-xr-x  4 root  wheel      128 May 28 01:56:42 2022 ..
-rw-r--r--  1 root  wheel        2 May 28 02:26:40 2022 builder_active%01
-rw-r--r--  1 root  wheel        2 May 28 02:31:24 2022 builder_active%02
-rw-r--r--  1 root  wheel        2 May 28 02:31:26 2022 builder_active%03
-rw-r--r--  1 root  wheel        2 May 28 02:31:26 2022 builder_active%04
-rw-r--r--  1 root  wheel       18 May 28 01:56:51 2022 originspec-pkgname%accessibility_accerciser
-rw-r--r--  1 root  wheel       19 May 28 01:56:51 2022 originspec-pkgname%accessibility_at-spi2-atk
-rw-r--r--  1 root  wheel       20 May 28 01:56:51 2022 originspec-pkgname%accessibility_at-spi2-core
-rw-r--r--  1 root  wheel       11 May 28 01:56:51 2022 originspec-pkgname%accessibility_atk
-rw-r--r--  1 root  wheel       13 May 28 01:56:51 2022 originspec-pkgname%accessibility_atkmm
-rw-r--r--  1 root  wheel       17 May 28 01:56:51 2022 originspec-pkgname%accessibility_caribou
. . .
-rw-r--r--  1 root  wheel       17 May 28 01:56:55 2022 pkgname-originspec%zutils-1.11
-rw-r--r--  1 root  wheel       15 May 28 02:03:32 2022 pkgname-originspec%zxfer-1.1.7
-rw-r--r--  1 root  wheel       14 May 28 02:03:00 2022 pkgname-originspec%zxid-1.42_1
-rw-r--r--  1 root  wheel       19 May 28 02:04:03 2022 pkgname-originspec%zxing-cpp-1.3.0
-rw-r--r--  1 root  wheel       12 May 28 01:59:41 2022 pkgname-originspec%zydis-3.1.0
-rw-r--r--  1 root  wheel       18 May 28 01:57:13 2022 pkgname-originspec%zynaddsubfx-3.0.6,2
-rw-r--r--  1 root  wheel        9 May 28 02:02:07 2022 pkgname-originspec%zyre-2.0.1
-rw-r--r--  1 root  wheel       14 May 28 01:59:41 2022 pkgname-originspec%zziplib-0.13.72_1
-rw-r--r--  1 root  wheel       14 May 28 02:03:00 2022 pkgname-originspec%zzuf-0.13_1
-rw-r--r--  1 root  wheel       13 May 28 01:56:45 2022 ports_metadata%top_git_hash
-rw-r--r--  1 root  wheel        4 May 28 01:56:49 2022 ports_metadata%top_unclean

Each file seems to have a 4096 Byte page used, making for
something around 320 MiBytes. Much of it not being heavily
used, some of it is gradually paged out to make room for
other things.

# du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/
326	/usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/

(Still no reported swap usage.)

For reference:

# du -xsAm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/
46	/usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/


===
Mark Millard
marklmi at yahoo.com