I've started collecting tmpfs usage figures from a poudriere-devel bulk -a for later publishing some of the top ones (handy for TMPFS_BLACKLIST judgments)
- Reply: Mark Millard : "Re: I've started collecting tmpfs usage figures from a poudriere-devel bulk -a for later publishing some of the top ones (handy for TMPFS_BLACKLIST judgments)"
- Reply: Tatsuki Makino : "Re: I've started collecting tmpfs usage figures from a poudriere-devel bulk -a for later publishing some of the top ones (handy for TMPFS_BLACKLIST judgments)"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Mon, 29 Apr 2024 01:48:35 UTC
I've modified my local poudriere-devel to have Success and Failure lines also report the tmpfs size at that point. Using, say, script to log the output to a file allows later sorting and listing of the TMPFS usage filgures. (The context is an amd64 one that has the resources to do a full bulk -a with USE_TMPFS=all invovled.) An example (from an in-progress bulk -a that has a long way to go): # grep TMPFS: ~/mmjnk-bulk-a-output.txt | sort -n -r -k11 [00:36:37] [18] [00:26:53] Finished print/texlive-docs | texlive-docs-20230313: Success ending TMPFS: 10.67 GiB [01:04:41] [07] [00:06:58] Finished net-mgmt/telegraf | telegraf-1.30.1_1: Success ending TMPFS: 10.52 GiB [01:03:32] [25] [00:06:09] Finished security/trivy | trivy-0.50.1_1: Success ending TMPFS: 10.10 GiB . . . [01:15:56] [20] [00:00:54] Finished databases/pg_tileserv | pg_tileserv-1.0.9_12: Failed: build TMPFS: 2.61 GiB . . . Note that the design is for sort with -k11 to work for Success and for Failure. (This is why "ending" is in place for Success.) I choose to use poudriere -N (no coloring) for this kind of activity. This helps for figuring out what all is appropriate for listing in TMPFS_BLACKLIST for a poudriere-devel configuration to avoid tmpfs competing too much for RAM+SWAP. (But approraite free file system space is needed.) "<" below is what is new, ">" is what was original, in /usr/local/share/poudriere/common.sh : 5928,5934d5927 < tmpfs_at_end="$(env BLOCKSIZE=512 df -t tmpfs \ < ${MASTERMNTROOT}/${MY_JOBID}/ \ < ${MASTERMNTROOT}/${MY_JOBID}/.p/ \ < ${MASTERMNTROOT}/${MY_JOBID}/usr/local/ \ < 2>/dev/null | tail -3 \ < | awk '{ tmpfs_use += $3; } END { printf "%s %.2f %s", "TMPFS:", tmpfs_use*512/(1024**3), "GiB" }')" < 5942c5935 < "Success${COLOR_RESET} ending ${tmpfs_at_end}" --- > "Success" 5968c5961 < "Failed: ${COLOR_PHASE}${failed_phase}${COLOR_RESET} ${tmpfs_at_end}" --- > "Failed: ${COLOR_PHASE}${failed_phase}" The form of use that I've done also involves (over?) use of MUTUALLY_EXCLUSIVE_BUILD_PACKAGES . It is not as good of data for this other use, but the same .txt file can be processed with: # grep TMPFS: ~/mmjnk-bulk-a-output.txt | sort -r -k3 | more [01:42:09] [04] [00:48:16] Finished lang/erlang-runtime21 | erlang-runtime21-21.3.8.24_3: Success ending TMPFS: 1.92 GiB [01:38:39] [28] [00:44:41] Finished lang/erlang-runtime22 | erlang-runtime22-22.3.4.27: Success ending TMPFS: 1.92 GiB [01:05:41] [02] [00:34:54] Finished lang/erlang-runtime26 | erlang-runtime26-26.2.4: Success ending TMPFS: 2.02 GiB . . . to find longer running package builds. This is subject to significant variation based on what other builders are running in parallel at the time and what sort of load averages are involved over period in question. The MUTUALLY_EXCLUSIVE_BUILD_PACKAGES that I've used will limit that to some extent. But the result is comparisons of some builds that have no activity in parallel by other builders vs. builds that have extensive parallel activity by other builders (of not-huge packages). Note: In modern times the [1D: notation and the like for what the -k3 compares are not well placed in the overall list compared to the likes of, say, [20: . The day vs. hour comparison is not a straight forward thing to sort on. Hopefully in a few days I'll be able to list off example top tmpfs usage for USE_TMPFS=all and top build times as well (such as they are). Note: This is from my personal environment. I've not tried to simulate how FreeBSD's official package builders are set up to operate. === Mark Millard marklmi at yahoo.com