(Retitled!) /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p (example) USE_TMPFS="data" tmpfs usage for poudriere
Date: Sat, 11 Jun 2022 22:03:26 UTC
On 2022-Jun-11, at 13:50, Mark Millard <marklmi@yahoo.com> wrote: > [I've deleted prior parts of the exchange.] > > I finally have a hypothesis with some evidence about what > primarily contributes to the 400 or so "1M-blocks Used" in: > > # df -mi /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/ > Filesystem 1M-blocks Used Avail Capacity iused ifree %iused Mounted on > tmpfs 1024 400 623 39% 103659 3828501 3% /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p > > --where the poudriere-devel is using USE_TMPFS="data" for a > "bulk -a -c" (that has been running for a little over 2 weeks > now and is past 1/3 done). > > Taking a quick estimate of the file count in . . ./ref/.p/var/cache/ > I get: > > # ls -Tla /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ | wc > 82146 821452 7942485 > > Taking 82146*4096 I get: 336,470,016 > Taking 336,470,016 / 1024 / 1024 I get around: 320 Mi > > By contrast: > > # du -xsAm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ > 46 /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ > > But: > > # du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ > 295 /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ > > and: > > # du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/ > 319 /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/ I see that a little relevant material you have not been able to see because of what I omitted from earlier messages --messages not sent to you at the time. In my earlier messages I'd shown: # du -xsAm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p 101 /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p # du -xsm /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p 68 /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p so -xsm resulsts were smaller than -xsAm results and way smaller than the "df -m" results. The 101-ish -xsAm result for .p/ has stayed fairly consistent. It appears that the swapoff activity mentioned below changed the -xsm results by forcing tmpfs material back into RAM. This ends up explaining the original 403 MiBytes vs. 101 MiBytes question that was mentioned in the original message subject. I mention this as it would appear to be important to trying to replicate or otherwise test the issue: Context matters for interpreting "du -xsm" results for tmpfs from what I can tell. > This is all after having done a swapoff for the 2 swap partitions > and a swap on for both and then waiting a while with the "bulk -a -c" > still running throughout and after. (So under 82 MiBytes is in > use in swap space at this point.) The above seems to be what lead to my "du -xsm" result being fairly close to the "df -m" result. > poudriere's tmpfs usage reporting does not seem to cover this > area's tmpfs usage. But, at least for "bulk -a -c" kinds of > activity, it ends up being the majority of the tmpfs usage for > USE_TMPFS="data" types of configuration. > > For reference: > > # ls -Tla /usr/local/poudriere/data/.m/main-CA7-bulk_a-default/ref/.p/var/cache/ | more > total 301234 > drwxr-xr-x 2 root wheel 5257152 May 28 02:31:26 2022 . > drwxr-xr-x 4 root wheel 128 May 28 01:56:42 2022 .. > -rw-r--r-- 1 root wheel 2 May 28 02:26:40 2022 builder_active%01 > -rw-r--r-- 1 root wheel 2 May 28 02:31:24 2022 builder_active%02 > -rw-r--r-- 1 root wheel 2 May 28 02:31:26 2022 builder_active%03 > -rw-r--r-- 1 root wheel 2 May 28 02:31:26 2022 builder_active%04 > -rw-r--r-- 1 root wheel 18 May 28 01:56:51 2022 originspec-pkgname%accessibility_accerciser > -rw-r--r-- 1 root wheel 19 May 28 01:56:51 2022 originspec-pkgname%accessibility_at-spi2-atk > -rw-r--r-- 1 root wheel 20 May 28 01:56:51 2022 originspec-pkgname%accessibility_at-spi2-core > -rw-r--r-- 1 root wheel 11 May 28 01:56:51 2022 originspec-pkgname%accessibility_atk > -rw-r--r-- 1 root wheel 13 May 28 01:56:51 2022 originspec-pkgname%accessibility_atkmm > -rw-r--r-- 1 root wheel 17 May 28 01:56:51 2022 originspec-pkgname%accessibility_caribou > . . . > -rw-r--r-- 1 root wheel 17 May 28 01:56:55 2022 pkgname-originspec%zutils-1.11 > -rw-r--r-- 1 root wheel 15 May 28 02:03:32 2022 pkgname-originspec%zxfer-1.1.7 > -rw-r--r-- 1 root wheel 14 May 28 02:03:00 2022 pkgname-originspec%zxid-1.42_1 > -rw-r--r-- 1 root wheel 19 May 28 02:04:03 2022 pkgname-originspec%zxing-cpp-1.3.0 > -rw-r--r-- 1 root wheel 12 May 28 01:59:41 2022 pkgname-originspec%zydis-3.1.0 > -rw-r--r-- 1 root wheel 18 May 28 01:57:13 2022 pkgname-originspec%zynaddsubfx-3.0.6,2 > -rw-r--r-- 1 root wheel 9 May 28 02:02:07 2022 pkgname-originspec%zyre-2.0.1 > -rw-r--r-- 1 root wheel 14 May 28 01:59:41 2022 pkgname-originspec%zziplib-0.13.72_1 > -rw-r--r-- 1 root wheel 14 May 28 02:03:00 2022 pkgname-originspec%zzuf-0.13_1 > -rw-r--r-- 1 root wheel 13 May 28 01:56:45 2022 ports_metadata%top_git_hash > -rw-r--r-- 1 root wheel 4 May 28 01:56:49 2022 ports_metadata%top_unclean === Mark Millard marklmi at yahoo.com