From nobody Mon Jul 10 17:31:18 2023 X-Original-To: freebsd-ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4R09zG0gzcz4dNJq for ; Mon, 10 Jul 2023 17:31:38 +0000 (UTC) (envelope-from rhurlin@gwdg.de) Received: from mailer.gwdg.de (mailer.gwdg.de [134.76.10.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4R09zF1cRHz3kPw for ; Mon, 10 Jul 2023 17:31:36 +0000 (UTC) (envelope-from rhurlin@gwdg.de) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of rhurlin@gwdg.de designates 134.76.10.26 as permitted sender) smtp.mailfrom=rhurlin@gwdg.de; dmarc=none Received: from excmbx-24.um.gwdg.de ([134.76.9.234] helo=email.gwdg.de) by mailer.gwdg.de with esmtp (GWDG Mailer) (envelope-from ) id 1qIujW-000CFM-UO; Mon, 10 Jul 2023 19:31:34 +0200 Received: from MBX19-GWD-03.um.gwdg.de (10.108.142.56) by excmbx-24.um.gwdg.de (134.76.9.234) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.2507.27; Mon, 10 Jul 2023 19:31:34 +0200 Received: from [192.168.178.23] (10.250.9.199) by MBX19-GWD-03.um.gwdg.de (10.108.142.56) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.1118.30; Mon, 10 Jul 2023 19:31:34 +0200 Message-ID: <2ea168aa-3e0e-ee31-d4a3-82b4ce46b330@gwdg.de> Date: Mon, 10 Jul 2023 19:31:18 +0200 List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: Problem with the package builds Content-Language: en-US From: Rainer Hurling To: Mark Millard CC: Konstantin Belousov , Kevin Oberman , FreeBSD Mailing List Reply-To: "Hurling, Rainer" References: <2DCC0132-0307-4A73-A473-B749ABF87367.ref@yahoo.com> <2DCC0132-0307-4A73-A473-B749ABF87367@yahoo.com> <1aaeafcc-9812-64bd-a369-bae7b9fc0e5f@gwdg.de> <422DBC3B-D85F-4AFB-ABDE-842A08482EC8@yahoo.com> <58250d08-ce0a-c1ac-ed15-7d55d517218e@gwdg.de> In-Reply-To: <58250d08-ce0a-c1ac-ed15-7d55d517218e@gwdg.de> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.250.9.199] X-ClientProxiedBy: excmbx-14.um.gwdg.de (134.76.9.225) To MBX19-GWD-03.um.gwdg.de (10.108.142.56) X-Virus-Scanned: (clean) by clamav X-Spamd-Result: default: False [-4.10 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; RCVD_IN_DNSWL_MED(-0.40)[134.76.10.26:from,134.76.9.234:received]; RWL_MAILSPIKE_EXCELLENT(-0.40)[134.76.10.26:from]; R_SPF_ALLOW(-0.20)[+ip4:134.76.10.0/23]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; MLMMJ_DEST(0.00)[freebsd-ports@freebsd.org]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; DMARC_NA(0.00)[gwdg.de]; FREEFALL_USER(0.00)[rhurlin]; HAS_REPLYTO(0.00)[rhurlin@FreeBSD.org]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; HAS_XOIP(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_TO(0.00)[yahoo.com]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:680, ipnet:134.76.0.0/16, country:DE]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[] X-Rspamd-Queue-Id: 4R09zF1cRHz3kPw X-Spamd-Bar: ---- X-ThisMailContainsUnwantedMimeParts: N Am 10.07.23 um 08:15 schrieb Rainer Hurling: > Am 10.07.23 um 08:08 schrieb Mark Millard: >> On Jul 9, 2023, at 22:52, Rainer Hurling wrote: >> >>> Am 10.07.23 um 06:53 schrieb Mark Millard: >>>> Konstantin Belousov wrote on >>>> Date: Mon, 10 Jul 2023 01:18:05 UTC : >>>>> On Sun, Jul 09, 2023 at 05:52:50PM -0700, Kevin Oberman wrote: >>>>>> Something looks really messed up on the beefy16 package builder for >>>>>> 13.1amd64. >>>>>> >>>>>> It is building about 9500 packages and started almost two days >>>>>> ago. It has >>>>>> only completed 459 packages, a rate of 10 packages/hour and, if it >>>>>> continues at this rate, will not complete for about 40 days. Nt >>>>>> sure if >>>>>> this was triggered by the recent updated to pkg, something that >>>>>> may have >>>>>> changed in poudriere or in the configuration, but almost all jails >>>>>> are are >>>>>> in one of the depends states, lib-depends, build-depends, >>>>>> run-depends, >>>>>> often running for hours. >>>>>> E.g. gnome-desktop-42.4_1 is in lib_depends and has been building for >>>>>> 1:09:59. Last build of the port took about 55 minutes. ATM there >>>>>> are 7 >>>>>> lib-depends builds, 5 build-depends, and 1 run-depends building. >>>>>> No jail is >>>>>> actually boulding the package, itself. >>>>>> >>>>>> Wild guess... It is rebuilding all depends every time. In any >>>>>> case, someone >>>>>> needs to look at this. >>>>>> >>>>>> Build report is at >>>>>> http://beefy16.nyi.freebsd.org/build.html?mastername=131amd64-default&build=e81dafc952d7 >>>>> >>>>> I noted that 'pkg-static install' seems to be much slower locally. >>>>> Not looked into it. >>>> Summarizing the later observations: >>>> Slow: pre-1400000 Jail OSVERSION and recent ports (such as pkg) >>>> ("default") >>>> Fast: pre-1400000 Jail OSVERSION and older ports  (such as pkg) >>>> ("quarterly") >>>> Fast: 1400093     Jail OSVERSION and recent ports (such as pkg) >>>> ("default") >>>> (Host OSVERSION 1400089 is a common context across them all.) >>>> Details: >>>> The list of build contexts with 50+ hr Elapsed but under 500 >>>> ports built looks like: >>>> All started on Sat, 08 Jul 2023 01:0[12]:?? GMT: >>>> beefy6 124amd64 default >>>> Host OSVERSION: 1400089 >>>> Jail OSVERSION: 1204000 >>>> beefy15 131i386 default >>>> Host OSVERSION: 1400089 >>>> Jail OSVERSION: 1301000 >>>> beefy16 131amd64 default >>>> Host OSVERSION: 1400089 >>>> Jail OSVERSION: 1301000 >>>> But the following 2 built over 5700 ports in under 50 hr Elapsed. >>>> Both started on Sat, 08 Jul 2023 01:01:?? GMT: >>>> beefy14 131amd64 quarterly >>>> Host OSVERSION: 1400089 >>>> Jail OSVERSION: 1301000 >>>> beefy13 131i386 quarterly >>>> Jail OSVERSION: 1301000 >>>> Ignoring armv6's qemu based builds, the more recently started >>>> main-amd64 and main-arm64 builds do not show the problem. >>>> Started Sat, 08 Jul 2023 01:29:09 GMT and built over 4400 ports >>>> in under 51 hr: >>>> beefy18 main-amd64 default >>>> Host OSVERSION: 1400089 >>>> Jail OSVERSION: 1400093 >>>> Started Sun, 09 Jul 2023 03:11:16 GMT 25:00:07 GMT and built over >>>> 1500 ports in under 26 hrs: >>>> ampere2 main-arm64 default >>>> Host OSVERSION: 1400089 >>>> Jail OSVERSION: 1400093 >>>> === >>>> Mark Millard >>>> marklmi at yahoo.com >>> >>> Since this does not seem to be mentioned in the previous posts. The >>> phenomenon does not only occur in the pkg clusters. >>> >>> In fact, I also observe this severely delayed behavior under >>> Poudriere on my very current 14.0-CURRENT box. All *-depends sections >>> need a multiple of the usual times ... >>> >> >> So, something like: >> >> Host OSVERSION: 1400093 >> Jail OSVERSION: 1400093 >> and: >> /packages/All/pkg-1.20.1_1.pkg > > Yes, this corresponds exactly to my versions of the system, the jail and > ports-mgmt/pkg. The newest update of ports-mgmt/pkg to version 1.20.2 by bapt@ solves the problem for me :) > >> >> is shown in the port build log files? That would invalidate >> the utility of the limited observations behind my earlier >> listing: >> >> Fast: 1400093     Jail OSVERSION and recent ports (such as pkg) >> ("default") >> >> Something else would have to contribute to the distinctions. >> >> >> === >> Mark Millard >> marklmi at yahoo.com >> > >