poudriere bulk -a fails on UFS: "Too many links" under logs/bulk/latest-per-pkg/ and then "Failed: starting"
Date: Sat, 04 Nov 2023 12:00:49 UTC
Looks like there are too many ports for "poudiere bulk -a" to work on UFS:
more than 32767 directory files to go in the likes of logs/bulk/latest-per-pkg/
as an example.
Context:
# uname -apKU
you have mail
FreeBSD amd64-UFS 15.0-CURRENT FreeBSD 15.0-CURRENT #126 main-n266130-d521abdff236-dirty: Tue Oct 24 18:17:40 PDT 2023 root@amd64-ZFS:/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG amd64 amd64 1500002 1500002
=> 40 2930277088 nda1 GPT (1.4T)
40 2008 - free - (1.0M)
2048 532480 1 efi (260M)
534528 1562624 - free - (763M)
2097152 2919235584 2 freebsd-ufs (1.4T)
2921332736 8944392 - free - (4.3G)
# ~/fbsd-based-on-what-commit.sh -C /usr/ports
6ec8e3450b29 (HEAD -> main, freebsd/main, freebsd/HEAD) devel/sdts++: Mark DEPRECATED
Author: Muhammad Moinur Rahman <bofh@FreeBSD.org>
Commit: Muhammad Moinur Rahman <bofh@FreeBSD.org>
CommitDate: 2023-10-21 19:01:38 +0000
branch: main
merge-base: 6ec8e3450b29462a590d09fb0b07ed214d456bd5
merge-base: CommitDate: 2023-10-21 19:01:38 +0000
n637598 (--first-parent --count for merge-base)
# ls -a /usr/local/poudriere/data/logs/bulk/latest-per-pkg/ | wc -l
32767
From the bulk -a output:
. . .
[30:24:09] [32] [00:00:42] Finished devel/p5-DateTime-Calendar-Hebrew | p5-DateTime-Calendar-Hebrew-0.05_1: Success
[30:24:10] [32] [00:00:00] Building dns/knot3 | knot3-3.3.1
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/knot3: Too many links
[30:24:13] [23] [00:00:22] Finished sysutils/ttyload | ttyload-0.5.3_1: Success
[30:24:14] [05] [00:01:44] Finished deskutils/mytetra | mytetra-1.43.27: Success
[30:24:15] [23] [00:00:01] Building audio/komposter | komposter-g20201211_1
[30:24:16] [05] [00:00:00] Building devel/pear-SebastianBergmann_PHPCPD | php81-pear-SebastianBergmann_PHPCPD-2.0.0
[30:24:16] [08] [00:18:43] Finished mail/cyrus-imapd36@http | cyrus-imapd36-http-3.6.3: Success
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/komposter: Too many links
[30:24:17] [08] [00:00:00] Building sysutils/p5-Samba-SIDhelper | p5-Samba-SIDhelper-0.0.0_3
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/php81-pear-SebastianBergmann_PHPCPD: Too many links
[30:24:18] [19] [00:00:38] Finished devel/p5-Test-WWW-Mechanize-CGIApp | p5-Test-WWW-Mechanize-CGIApp-0.05_1: Success
[30:24:18] [12] [00:00:24] Finished net-mgmt/p5-Mon | p5-Mon-0.11_1: Success
[30:24:19] [19] [00:00:00] Building mail/squirrelmail-plugins@php82 | squirrelmail-plugins-php82-1.0_2
[30:24:19] [12] [00:00:00] Building security/bsmtrace | bsmtrace-1.4_1
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/p5-Samba-SIDhelper: Too many links
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/squirrelmail-plugins-php82: Too many links
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/bsmtrace: Too many links
[30:24:24] [17] [00:00:29] Finished devel/p5-Class-Delegation | p5-Class-Delegation-1.9.0: Success
[30:24:24] [17] [00:00:00] Building devel/py-memory-profiler@py39 | py39-memory-profiler-0.61.0
[30:24:25] [13] [00:00:21] Finished converters/nomyso | nomyso-4.3: Success
[30:24:25] [13] [00:00:00] Building devel/rubygem-aws-sdk | rubygem-aws-sdk-3.1.0
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/py39-memory-profiler: Too many links
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/rubygem-aws-sdk: Too many links
[30:24:29] [16] [00:15:52] Finished games/sauerbraten | sauerbraten-20201221_2: Success
[30:24:30] [16] [00:00:00] Building lang/ccl | ccl-1.12_2
[30:24:30] [20] [00:09:58] Finished security/sssd@default | sssd-1.16.5_10: Success
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/ccl: Too many links
[30:24:33] [20] [00:00:00] Building games/xpired | xpired-1.22_24
[30:24:33] [18] [00:01:12] Finished irc/miau | miau-0.6.6_3: Success
[30:24:34] [18] [00:00:00] Building textproc/rubygem-nokogumbo | rubygem-nokogumbo-2.0.5_2
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/xpired: Too many links
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/rubygem-nokogumbo: Too many links
[30:24:39] [22] [00:01:12] Finished net/zsync | zsync-0.6.2_2: Success
[30:24:39] [22] [00:00:00] Building devel/distcc | distcc-3.4
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/distcc: Too many links
[30:24:49] [01] [00:05:11] Finished audio/myxer | myxer-1.2.1_26: Success
[30:24:50] [01] [00:00:00] Building security/theonionbox | theonionbox-4.3.1_2
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/theonionbox: Too many links
[30:24:56] [21] [00:01:41] Finished dns/knock | knockpy-5.4.0_2: Success
[30:24:57] [29] [00:01:56] Finished games/meandmyshadow | meandmyshadow-0.5a_2: Success
[30:24:58] [21] [00:00:00] Building biology/phyml | phyml-3.3.20220408_1,1
[30:24:58] [29] [00:00:00] Building security/globalprotect-openconnect | globalprotect-openconnect-1.4.9_1
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/phyml: Too many links
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/globalprotect-openconnect: Too many links
[30:25:00] [06] [00:01:21] Finished www/mod_auth_openidc | ap24-mod_auth_openidc-2.4.14.2: Success
[30:25:00] [06] [00:00:00] Building net-mgmt/torrus | torrus-2.09_2
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/torrus: Too many links
[30:25:03] [24] [00:02:29] Finished www/dooble@qt5 | dooble-qt5-2023.08.30: Success
[30:25:03] [24] [00:00:00] Building games/openarena-oax | openarena-oax-B51_1
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/openarena-oax: Too many links
[30:25:19] [28] [00:02:11] Finished www/npm-node20 | npm-node20-10.2.0: Success
[30:25:20] [28] [00:00:00] Building editors/hexcurse | hexcurse-1.60.0
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/hexcurse: Too many links
[30:25:27] [25] [00:16:04] Finished mail/cyrus-imapd34@http | cyrus-imapd34-http-3.4.6: Success
[30:25:27] [25] [00:00:00] Building www/rustypaste-cli | rustypaste-cli-0.8.0
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/rustypaste-cli: Too many links
[30:25:39] [10] [00:01:34] Finished misc/lightgbm | lightgbm-4.1.0: Success
[30:25:40] [10] [00:00:00] Building audio/mkcue | mkcue-1
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/mkcue: Too many links
[30:26:10] [01] [30:26:10] Finished security/theonionbox | theonionbox-4.3.1_2: Failed: starting
[30:26:11] [05] [30:26:11] Finished devel/pear-SebastianBergmann_PHPCPD | php81-pear-SebastianBergmann_PHPCPD-2.0.0: Failed: starting
[30:26:11] [01] [00:00:00] Building editors/openoffice-4 | apache-openoffice-4.1.14_3
[30:26:12] [06] [30:26:12] Finished net-mgmt/torrus | torrus-2.09_2: Failed: starting
[30:26:12] [05] [00:00:00] Building databases/pgreplay | pgreplay-1.3.0_2
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/apache-openoffice: Too many links
[30:26:13] [08] [30:26:13] Finished sysutils/p5-Samba-SIDhelper | p5-Samba-SIDhelper-0.0.0_3: Failed: starting
[30:26:13] [06] [00:00:00] Building cad/qflow | qflow-1.4.100_1
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/pgreplay: Too many links
[30:26:14] [10] [30:26:14] Finished audio/mkcue | mkcue-1: Failed: starting
[30:26:14] [08] [00:00:00] Building sysutils/mcweject | mcweject-1.1
[30:26:14] [10] [00:00:00] Building devel/pear-Net_Gearman@php83 | php83-pear-Net_Gearman-0.2.3_3
[30:26:14] [12] [30:26:14] Finished security/bsmtrace | bsmtrace-1.4_1: Failed: starting
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/qflow: Too many links
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/mcweject: Too many links
[30:26:15] [13] [30:26:15] Finished devel/rubygem-aws-sdk | rubygem-aws-sdk-3.1.0: Failed: starting
[30:26:15] [12] [00:00:00] Building net/cloud-init-devel@py39 | py39-cloud-init-devel-23.3.1
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/php83-pear-Net_Gearman: Too many links
[30:26:16] [11] [00:02:11] Finished japanese/font-plemoljp | ja-font-plemoljp-1.6.0: Success
[30:26:16] [16] [30:26:16] Finished lang/ccl | ccl-1.12_2: Failed: starting
[30:26:16] [13] [00:00:00] Building devel/py-pyupgrade@py39 | py39-pyupgrade-3.15.0
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/py39-cloud-init-devel: Too many links
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/py39-pyupgrade: Too many links
[30:26:18] [16] [00:00:00] Building lang/frawk | frawk-0.4.7
[30:26:18] [17] [30:26:18] Finished devel/py-memory-profiler@py39 | py39-memory-profiler-0.61.0: Failed: starting
[30:26:19] [18] [30:26:19] Finished textproc/rubygem-nokogumbo | rubygem-nokogumbo-2.0.5_2: Failed: starting
[30:26:19] [17] [00:00:00] Building net/rubygem-google-cloud-spanner | rubygem-google-cloud-spanner-2.18.1
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/frawk: Too many links
mkdir: /usr/local/poudriere/data/logs/bulk/latest-per-pkg/rubygem-google-cloud-spanner: Too many links
[30:26:20] [18] [00:00:00] Building www/p5-HTML-Widgets-SelectLayers | p5-HTML-Widgets-SelectLayers-0.07_2
[30:26:20] [19] [30:26:20] Finished mail/squirrelmail-plugins@php82 | squirrelmail-plugins-php82-1.0_2: Failed: starting
. . .
# poudriere status -b
[main-amd64-bulk_a-default] [2023-11-02_21h44m56s] [parallel_build:] Queued: 34683 Built: 32285 Failed: 1038 Skipped: 366 Ignored: 320 Fetched: 0 Tobuild: 674 Time: 30:50:53
ID TOTAL ORIGIN PKGNAME PHASE PHASE TMPFS CPU% MEM%
[29] 00:00:30 misc/swissfileknife | swissfileknife-1.9.9.0 starting 00:00:30 1.28 GiB [15] 00:52:19 cad/kicad-library-packages3d | kicad-library-packages3d-7.0.2_2 package 00:48:06 16.55 GiB 100% 0.1%
[30] 00:00:30 sysutils/mac_rtprio | mac_rtprio-kmod-g20170417 starting 00:00:30 1.28 GiB [01] 00:00:00 net/nakenchat | nakenchat-3.00.b1 starting 00:00:00 1.43 GiB [16] 00:00:31 ports-mgmt/pkgs_which | pkgs_which-0.4.1 starting 00:00:31 1.28 GiB [31] 00:00:30 x11-themes/Kvantum | Kvantum-qt5-1.0.10 starting 00:00:30 1.28 GiB [02] 00:00:00 graphics/pear-Image_Barcode@php83 | php83-pear-Image_Barcode-1.1.3 starting 00:00:00 1.43 GiB [17] 00:00:31 games/egoboo | egoboo-2.8.1_1,1 starting 00:00:31 1.28 GiB [32] 00:00:30 textproc/fa-aspell | fa-aspell-0.11.0_1,2 starting 00:00:30 1.28 GiB [03] 00:00:00 comms/sigdigger | sigdigger-0.3.0.1_1 starting 00:00:00 1.39 GiB [18] 00:00:31 devel/py-pdoc@py39 | py39-pdoc-14.1.0 starting 00:00:31 1.28 GiB [04] 00:00:00 graphics/pecl-qrencode@php80 | php80-pecl-qrencode-0.11 starting 00:00:00 1.31 GiB [19] 00:00:31 devel/websvn@php81 | websvn-php81-2.8.2_1 starting 00:00:31 1.28 GiB [05] 00:00:00 mail/squirrelmail-quota_usage-plugin@php82 | squirrelmail-quota_usage-plugin-php82-1.3.1_3 starting 00:00:00 1.28 GiB [20] 00:00:31 textproc/p5-Text-DHCPLeases | p5-Text-DHCPLeases-1.0_1 starting 00:00:31 1.28 GiB [06] 00:00:00 textproc/html2fo | html2fo-0.4.2 starting 00:00:00 1.29 GiB [21] 00:00:31 graphics/partio | partio-1.14.6_2 starting 00:00:31 1.28 GiB [07] 00:00:00 math/py-pdal@py39 | py39-pdal-3.0.2_2 starting 00:00:00 1.28 GiB [22] 00:00:31 cad/cura | Cura-4.13.1_4,2 starting 00:00:31 1.28 GiB [08] 00:00:00 mail/mboxstats | mboxstats-3.1 starting 00:00:00 1.28 GiB [23] 00:00:31 textproc/p5-Text-HikiDoc | p5-Text-HikiDoc-1.023 starting 00:00:31 1.28 GiB [09] crashed [24] 00:00:31 lang/zephir@php81 | php81-zephir-0.17.0 starting 00:00:31 1.28 GiB [10] 00:00:32 databases/php-xapian@php82 | php82-xapian-1.4.23 starting 00:00:32 1.28 GiB [25] 00:00:31 security/stegify | stegify-1.2.2_14 starting 00:00:31 1.28 GiB [11] 00:00:32 science/py-qspin@py39 | py39-qspin-2.3.3 starting 00:00:32 1.28 GiB [26] 02:08:19 emulators/libretro-mame | libretro-mame-20220124_1 build 02:06:29 3.67 GiB 0% 0.1%
[12] 00:00:32 devel/pear-PEAR_Info@php83 | php83-pear-PEAR_Info-1.9.2_4 starting 00:00:32 1.28 GiB [27] 00:00:30 www/moodle42@php82 | moodle42-php82-4.2.3 starting 00:00:30 1.28 GiB [13] 00:00:32 sysutils/luckybackup | luckybackup-0.5.0_2 starting 00:00:32 1.28 GiB [28] 00:00:30 news/mmail | mmail-0.52 starting 00:00:30 1.28 GiB [14] 00:00:32 math/giacxcas | giacxcas-1.9.0.55 starting 00:00:32 1.28 GiB =>> Logs: /usr/local/poudriere/data/logs/bulk/main-amd64-bulk_a-default/2023-11-02_21h44m56s
===
Mark Millard
marklmi at yahoo.com