From nobody Sun Nov 05 00:17:19 2023 X-Original-To: freebsd-ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SNFRm5QhSz50MBP for ; Sun, 5 Nov 2023 00:17:40 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-25.consmr.mail.gq1.yahoo.com (sonic304-25.consmr.mail.gq1.yahoo.com [98.137.68.206]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SNFRm2kz3z3QZj for ; Sun, 5 Nov 2023 00:17:40 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1699143457; bh=LFKKan6ta7lfbVfSaPrU9DFjyM5JcCeU8VVCDd+2WWM=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=MKj0nGTGG6fiBdaduwYm1pZ/jnlEWyyATHYf5boVM+JCuy217PGSE2ZCRCL+R/eDS2XTeLLAEkeL/5nuHrvoDIDudt0xXw62OnVypRba4o8s3Mo3wB8Q3znOxR0n4P5J8mSOeI6W01oVnMYIpPBBpxBo8sZVmODEx8MtCCrofMUPJ3Y1NppodFRo8MfaCHD7XhpenbnlV9nJfW4MWOznoahQRRoN+pb7CDnkLBqroybBI7gYW0Gvl6KSrqTxNIuTXxzy34zXuhL0ZXtV3xhXfWCksiNhsvx8qRfa7S5OiS5GbE4fTiCKzoTItSdzp/RtKLJhgqLt95L0PqMmEQwJOw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1699143457; bh=BR9Uq95oEY1GP1uqzWWy2uyP4lcRHXWcAJGzihJaFAc=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=Le6zyrankHMMX+Lpc5/P6IMbWU0Aa70L+U2NAwmBDGQfMj3HgAE83YBcJ7B2Vnp5Hputlsx3Cd2Axny7QpG1V5+dGrZMq3bJivS4Y8h2TQ5IVt25ymT7nqCQdBEvJ/BNdLmOO+3tDy2EBZTqIUsR5HPutoQMGseJA5i0XTcx1R4JUYvBiY6Dsk7C3jin1Q0+LPOWatGL7b8yu3xG3CW85bhm5fXCQEDTXKGzPCbYEgd9yAZYGKH2SDHlXeszZdfqIwyZ5gSGyUxOfRNtoOZyoAZylX0E8Vmju6FyINHeVbALqr67wLeHCC79s9ao2bGxZNVQtLiGE9LBg4QYBJGFxQ== X-YMail-OSG: 1Po8G2kVM1mtXiFmq2QO_sOk6nHR9HNmN3U4QQI2KEFRtrH6piIWDJU6MM8414A 6HAExjcX18bcECmTGqN6M_UxREYc5DG6rbFhu_kJsLtdeQxIaRuP355Sa1ixg4bJy3I.nxMnTIW7 kmD41bJq.4Ml6j7Cr3xyU6UN67.qg0hzrVHh6m7cOaQFbrsmuYu79rEljSZPyaMY1aRTjXc4l2b4 QsavqGpSCUHvGs6ZTMtVwF1CGbyLdcMcIioDDEQNJJUaHQoVfJWVSyIEiM4CGelhhwFq3RkpPdbr N8dTeUfBiQ.x3ieeMT17HCGIlFPyJrrAFp4SvBw8wLvQGwn8Pu1H.2b_tX3QO1bCLl0NCrVeTd6S B1s6z089UeIzGaIDUIFo4aBcfU1_AcOwU9ka8b1Q1tqgHVwbYvuYWUExkFU8xlo_eJXvUgGI8ziZ iGfcMxzJ0lEF7LZ2WTHF_0L6Lmty7mB2zrahdiynyja5yMBW.VOqRBYmrdYdqGz1k2JLk8pMoihy lB5rVWErpJ8l_OJznYimR7Ea5ArLZ3jhJ2sDS8uJmPf_2Hho4vCniAGF7P3_LcHh5jjtt_Rlss_w cm8n6IKAR9ELtTRzveZFqNmskE5hVTZKmMVT_CX9k1KO4OYboG8yVsXYi2z5Rsna8Ghk2soc4Ux_ 8ZAzv.rd5Xp_iI3CtrcGWQhJ8WQ38qo9Cr4wFLjRqyK81MrOEBwgffCS0ZDl09v3jQlT9YF5pDOO 8XHH8svJm9A6u3hCzorXVmsbadtkFBNOTF.H.GBFibXKrDuX_C1TcmK5q56Xq6yu_8KyXpsgjEQ7 69S9XXM6qTOrNS1TlU5Q5hCtVTcpSc3MWQtIK9k1UAyQNC3qce8ZlkVSulqMbngMBVwUoS18AuSo okQeQ54KuZIuRYeF_cQ6UKfVZ6GfpTVrwZ_eKfmvZ_sQe9XABK_.2aYF_cKlnnsP8uU7OIwLW4fO RH2azDDenK1qRVKe38XSqTM3Wqs2DPKt_mpNbq1iCDy5bpGvYPaeZFF47Sltnp2R0FyYfkuFBsGe SWhAfrsQn5YP4m5wBnrhJ25Jo.YbALjvOhoVM.1pvxtsNsbA7qvlRJlmwOnaAFyXd9PHOONeUfSq GnD_h4y8Zvm9Aqa3e1puGQyVV9YAeXdPKE_cjtbkFPszHffoq1u0CaKakuqvUjZoTF0ej16yKRQL ZVAnXvjObzr2R5AmzOfj.5iC0D.ZJu9Zxk4JZF72IX1WWZHRkb.xdj_diXqQyMpnrP2hSjxmBpUB ftefkAB_q9vRuTQVNduk9WITNlPmtAPsI7MmUBLHA3taQAlW8DaX7cnJQlIHJf8n_O8iOMAKFnQE P.12J8KLL040Wo970WX.JOL.Xxi3UQaUJwZoUVVvpNE_Q8V_nPwTXFsnq8EeMqVhxoe9JV5cvBsm 1oxF0QwpEOyjETTdPmtacBfYxqUyqIPOfR5Snvf30cSjkiRSuJkdW_G9c4fmpKRZ.NbvGUXttv6J 9Q4E_3HW3OJOC2tdQyJ3bVqxrAY5p_AkOZ75l9fYyODFmKnLYwtPQ0HpmDwdZiFzi4qZIADICX.0 Xiki9D0eLE7e9AhQH1bS9Ymitrfqh5UPvO9hcaBhD.GCbkHTCDDfz72Of0JVrBfpwQfg2QjuDWph YkdMHUyYoPj2d8WuAg9VJ0bPmM0fvK1.aVn0BIfAoMLt.eSHeyPSkAJ.dpZYsulJBpW4h2bquOB0 jTgUCU1ISgMTXdLwbrct5qJssjCD1n.YczK6AYB.P5EZ5tVrOLTDbJS5l6ni0EFuDGcLteDPaIjP sie6.HoMNm95WfHUNWCUoMZHNVxJ2bxb8epgsG6hsRXKDnY9vqCIQeuZ.icqiSWf7z.cP36VcCH_ 1JeUBLg8Z4OJnmOX0b83tCvpxwVqNi8PNJIeB_jKodkaggAhG0Oin_YelJaEsntKqGInpxrYX32g _K1GQwiu6BqUXyydfQ3PsT4FcYm8iN0RQsTT1.unNS5xLOrD_YmspA1JCnhMKUc59I6TX67QFykN e8EQEHHKBNv6hPHYCG1RXGPcjtqSwCWc05_kgk83_QRtd0dBJMU9rhYMzRFJAauIlqQdJlc6JSe2 ByiJAkA585A6YNIQA__gIF7ZIt5UVooR798jsV8GZyXmc_kUh9pSUctkoxELAyb_hW.0XdcxXEB8 RCA9d.IyOHGwE7yJqO4BRI5EwMS_SCpNqdoH1zFBhlo8SkXBdkOX_4s1717kRkP0bNMklWyxUQoi J3z.LqV_bEwm..tgyfv51MEZ3Tgp.s_QpnMalsLYPreLowWMoXiMvcX78F9hRUFCcjR6r9I5H5SN 7jg-- X-Sonic-MF: X-Sonic-ID: 36b7b1fe-63d9-47fa-a3f2-f428c555f5ba Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Sun, 5 Nov 2023 00:17:37 +0000 Received: by hermes--production-ne1-56df75844-5vnpc (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID fec125460052bf2499240c97e69f6116; Sun, 05 Nov 2023 00:17:31 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.200.91.1.1\)) Subject: Re: poudriere bulk -a fails on UFS: "Too many links" under logs/bulk/latest-per-pkg/ and then "Failed: starting" From: Mark Millard In-Reply-To: Date: Sat, 4 Nov 2023 17:17:19 -0700 Cc: "freebsd-ports@FreeBSD.org" Content-Transfer-Encoding: quoted-printable Message-Id: References: <17C21264-D87B-40D3-9228-579F7BC3DD7C@yahoo.com> To: Tatsuki Makino X-Mailer: Apple Mail (2.3774.200.91.1.1) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4SNFRm2kz3z3QZj On Nov 4, 2023, at 16:45, Tatsuki Makino = wrote: > Mark Millard wrote on 2023/11/05 07:54: >> (I've not checked if = /usr/local/poudriere/data/logs/bulk/latest-per-pkg/ >> is the only issue. It is just the first context to stop the build.) >=20 > Sorry, I didn't read it very carefully and confused the logs directory = with the packages directory :) > I think we can delete the old logs. > But just do the following. >=20 > poudriere logclean 30 # If the last 30 days are to be kept. Even if there are 0 logs before the bulk -a starts, the bulk -a will fail to finish on a UFS file system. This happens because there are over 32767 packages these days that will get logs for an amd64 context and poudriere chooses to organize its directory tree such that the UFS limitation will be hit. (Plus UFS has the limitation in the first place.) I'll also note that my bulk -a experiments are not important and would not justify changing anything. But they have lead to the issue being discovered and reported so that the limitation is better known now. > There is a -N count option in poudriere-logclean, but I think it would = cause less problems if it were not used. > Because awk is behaving strangely. > It occurs that logs with old dates continue to be kept. > It could also leave directories for packages whose names no longer = exist. > I don't know if awk behavior has been fixed in newer versions of = FreeBSD. > It is strange, at least in 12.x. >=20 None of this matters for the issue with what goes directly in /usr/local/poudriere/data/logs/bulk/latest-per-pkg/ The count will still attempt to become larger than 32767 in di_nlink in ufs2_dinode for that directory. UFS will refuse such attempts. I'll note that having a few thousand or more packages that never have an attempted build works fine on UFS. (Over time a larger subset must not be built in order to stay under 32768.) Also, the official FreeBSD package build servers use ZFS for the file system and, so, do not have the problem. I do not expect UFS to ever be changed to use more than 16 bits for the di_nlink purpose. I'd be surprised if poudriere changed its directory tree structure to avoid the issue on UFS, given that official builds work as-is. I expect that folks that want to do bulk -a runs will just have to set up a (probably) ZFS filesystem. This much should be documented if it stays as a requirement for bulk -a use to be successful. (And, my ZFS bulk -a experiment worked just fine, given that I'd configured well for my style of build.) =3D=3D=3D Mark Millard marklmi at yahoo.com