From nobody Sat Sep 14 23:38:43 2024 X-Original-To: freebsd-ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4X5ngh4Nywz5WS8D for ; Sat, 14 Sep 2024 23:38:56 +0000 (UTC) (envelope-from john.a.schneider@gmail.com) Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com [IPv6:2607:f8b0:4864:20::52f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4X5ngh0gzzz4r0j for ; Sat, 14 Sep 2024 23:38:56 +0000 (UTC) (envelope-from john.a.schneider@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pg1-x52f.google.com with SMTP id 41be03b00d2f7-7d50e7a3652so2192025a12.3 for ; Sat, 14 Sep 2024 16:38:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726357135; x=1726961935; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=GRXo16WhKvPTIuZVMXn22vVo7FJiVTU1EHD0efZl3io=; b=AHkwpMu+2h/F/dMwP4ru31e0cmUsHwNfHKXxZO/MSAk5Xflq3Z9M0GLFL6pK+V8kf4 h2DBpHtqu8we9y5oGv1ddNmPh9njP1q2VWWF6RUnHtCB6B3AQXl6w4f7pHS5c2uJy+/S Vx6VjhpCbDVGHBRu/tIrtGpc0g0eqTfFPJerOf2Ac3+hHncGnXj1Qxa4z7RDB/VyfWHi 6HkOYP3JuuQz3dCZ6Fk/QAtarHgg8oRgvHnyiJsfoCvhcziYasTPvyC1p/D0u7wDAaxC 6Ppx7wpahj3BxRRIUklYp1CWiqLG6AIWqjbxypUssnJx/7Dh2IifnA5jBjTvuuT7Py9v vfqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726357135; x=1726961935; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=GRXo16WhKvPTIuZVMXn22vVo7FJiVTU1EHD0efZl3io=; b=ij8Tgw6ar8sI8JB3X4Llqpr9k63sABqqLgmwQfzXcS6DVJt9TOrKlvY7McRaYBei35 u4+Gi5ckiCMAXLIQX8xdgcJ+4YGC1rarhy1ccrQJcqhBD7/v+R9Px126ionbFR3cBSWK jAG7FKVhJ/JsbIfGjy3pIgBIoIWy29681XmKT2USaeLx2KjeIewOsAJi81XLzxJus0QF W+FcWTLmHkQk850BDyL51Jm8eXABnIgpYwkDrFCqstYfdx1pILNausjetnPo87J/o7uS 7bFjjl2sapKGqdNwliJnnKgkxgkdL3KW0an5F1LZoFdG2ghqvmYHmkwe/ZLwzzQFidRP Y4kA== X-Gm-Message-State: AOJu0YzsiZnPh5KX3f8LrXZxdtFDqQD5tjPb8TqiqyuI/hRZkEhm/l/K kXoqkgTqA9v/SbohVTfhXNzX3hme7y9jZAp/EwbpaBjzjbln4APLFy6aBhColoaE2uNcnvaZ+zr cC5s9r8x41g1g+05nfI2WVDDBNF4y3E/O X-Google-Smtp-Source: AGHT+IHywdLyAYzaKTWWl12K7RL9zqM6PHUeaDWgp9eS9sL39YE8nLYb6l/+AJAGkKMejySFb9+6VGpZSg4PoEjoOnw= X-Received: by 2002:a05:6a21:1707:b0:1cf:43c6:d546 with SMTP id adf61e73a8af0-1cf75d7fe62mr16503884637.3.1726357134429; Sat, 14 Sep 2024 16:38:54 -0700 (PDT) List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-ports@freebsd.org Sender: owner-freebsd-ports@FreeBSD.org MIME-Version: 1.0 References: <8843FFE6-84C9-4DCE-8524-7B912D6C43DC.ref@yahoo.com> <8843FFE6-84C9-4DCE-8524-7B912D6C43DC@yahoo.com> <4EB71408-1A4D-4C05-BAED-8756EE6464ED@yahoo.com> In-Reply-To: <4EB71408-1A4D-4C05-BAED-8756EE6464ED@yahoo.com> From: John Schneider Date: Sat, 14 Sep 2024 19:38:43 -0400 Message-ID: Subject: Re: Official package builder poudriere.conf update? To: Mark Millard Cc: FreeBSD Mailing List Content-Type: multipart/alternative; boundary="000000000000c0d8cd06221cd7e9" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4X5ngh0gzzz4r0j --000000000000c0d8cd06221cd7e9 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Excellent. I'm glad poudriere has this change already ( https://github.com/freebsd/poudriere/commit/9ea59e59f12f2b77699de9cafa5ceea= 1f7814eaf#diff-afbea33008c9478b94f3379278925c57204b3283411000094ee48d238829= 3a5f ). I hadn't realized common.sh had been updated and I simply needed to take out the related config from my poudriere.conf I'm using 3.4.1_1. Thanks for the assistance. On Sat, Sep 14, 2024 at 7:05=E2=80=AFPM Mark Millard wr= ote: > As of 2023-Dec-19 or source when FreeBSD's poudriere on the official > builders > started being based on 3.4.0. So the MAX_FILES default is 8192, not 4096, > as > of that change. Numerous official builds succeed based on the default > value, > including just recently. > > If there is an explicit MAX_FILES_vscode=3D4096 somewhere that would use > poudriere > 3.4.0 or later, the assignment should simply be deleted (instead of > separately raised). > > What version of poudriere are you using? Is it based on 3.4.0 or later? > > Mark > > > On Sep 14, 2024, at 15:45, John Schneider > wrote: > > Hi Mark. Thanks for this very thorough explanation. I'm currently > successful at building electron via poudriere with multiple cores in a > reasonable amount of time, with no special config. The thing that was > biting me was the maximum number of files knob for vscode > (MAX_FILES_vscode) in poudriere.conf. I imagine we're really close to th= e > limit with value of 4096 at present. Perhaps sometimes 4K is enough and > with VSCode source code updates we go back and forth over this limit? > > I can currently reproduce the build error described in bug > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242871 a > MAX_FILES_vscode=3D4096 and the build is successful with a higher value. = I > don't know if a value of 8192 is appropriate or not but it definitely fix= es > the package build. > > Assuming we continue to have official stable package builds for electron, > I think the master poudriere config needs to change to have > MAX_FILES_vscode be raised. > > Thanks, > John > > On Fri, Sep 13, 2024 at 9:12=E2=80=AFPM Mark Millard = wrote: > >> John Schneider wrote on >> Date: Fri, 13 Sep 2024 21:58:52 UTC : >> >> > I've noticed the FreeBSD package mirrors (pkg.freebsd.org) haven't >> included >> > packages for editors/vscode. Reference: >> > http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/release_1/ >> >> http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/release_1/ is >> frozen as things were just before 14.1-RELEASE was made: >> back on 2024-May-19. It is never updated as far as I know. >> So: as-is, including what failed to be built at the time. >> It is what is incuded on the official release DVD's that >> have the built port packages. >> >> http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/release_0/ is >> similar but goes back to 2023-Oct-15. >> >> The port package repos for 14.x that are updated over time >> are: >> >> http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/quarterly/ >> http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/latest/ >> >> In my testing just now, use of: >> >> url: "pkg+https://pkg.FreeBSD.org/${ABI}/quarterly >> ", >> >> did not find vscode as stands. Later below I report >> on why/how. But use of: >> >> url: "pkg+https://pkg.FreeBSD.org/${ABI}/latest >> ", >> >> did find vscode. >> >> I used a chroot into a stable/14.1 context on a >> machine that booted main [so: 15] (both kernel and >> world). releng/14.* and stable/14 use the same >> pacakge repos via the ABI text also being the same >> in the URL's. . . . >> >> # file /bin/sh >> /bin/sh: ELF 64-bit LSB pie executable, x86-64, version 1 (FreeBSD), >> dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 14.1 >> (1401501), FreeBSD-style, stripped >> >> # pkg search vscode >> vscode-1.92.2 Visual Studio Code - Open Source ("Code - >> OSS") >> >> This was after a round of it synchronizing my context: >> >> # env ABI=3DFreeBSD:14:amd64 IGNORE_OSVERSION=3Dyes pkg search vscode >> pkg: Repository FreeBSD has a wrong packagesite, need to re-create >> database >> vscode-1.92.2 Visual Studio Code - Open Source ("Code - >> OSS") >> >> https://www.freshports.org/editors/vscode/ reports only 3 vscode >> builds: >> >> FreeBSD:13:latest has 1.92.2 >> FreeBSD:14:latest has 1.92.2 >> FreeBSD:15:latest has 1.92.2 >> >> Also, vscode is listed as depending on: devel/electron30 at >> this time. >> >> It is common for all but one devel/electron* to be >> disabled for pkg builds: "blacklisted". >> >> >> https://pkg-status.freebsd.org/beefy20/build.html?mastername=3D140amd64-= quarterly&build=3D59d8804dcdd7 >> >> (started 2024-Sep-05) reports that vscode was skipped because >> electron29-29.4.6 did not build at the time. Note that this >> is a September example of quarterly. >> >> In turn it reports that electron29-29.4.6 was an Ingored Port >> for the Reason: "Blacklisted". >> >> In essence the electron* 's are treated has too many using too >> many resources and time to build them all without other >> negative tradeoffs for the overall context. >> >> > Could this be because the official package vscode build is failing? >> >> I've only shown the one example. No claim that a electron* >> is the only way for vscode builds to fail. >> >> > I was >> > able to overcome a problem with too many file descriptors during the >> build >> > by changing my poudriere.conf file to double MAX_FILES_vscode=3D4096 t= o >> > MAX_FILES_vscode=3D8192 >> >> That will not help with the required electron* not being >> available. >> >> > This was previously addressed in bug 242871 - >> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D242871 >> > >> > If there's a better mailing list for this issue, please refer me to it= . >> >> >> I'll note that on the aarch64 (so: Tier 1) builder for >> main (ampere2) the following all fail to build after >> between 4hr and 49hr of attempting to build >> (build/timeout failures): >> >> www/ungoogled-chromium >> www/iridium >> www/chromium >> devel/electron30 >> >> For how things are configured, attempting to build those >> seems to mostly just be wasted resources and time for >> aarch64. Other devel/electron* would likely be the same. >> (Other configurations would have other tradeoffs and >> might not prove sufficient for the overall context.) >> >> I'll note that the next biggest time taker fails for >> other reasons in under 3hr. Also, the largest time >> taker that builds is www/qt6-webengine at between >> 27hr and 28hr currently. The next 2 builds by time >> take between 19hr and 20hr currently >> ( databases/mongodb80 and www/qt5-webengine ). >> >> There is a huge difference for building the various: >> >> www/ungoogled-chromium >> www/iridium >> www/chromium >> devel/electron* >> >> compared to anything else. (I'm not spanning >> prerequisites here: just the direct part of >> the overall build for each.) >> >> I've no clue what time limits would allow those to >> all build on the ampere*, leaving things configured >> the same otherwise. I also do not know what would >> happen if they all started building at about the >> same time, up to 13 builders, given 13 FreeBSD cpus >> in each ampere* . >> >> Anything based on a devel/electron* suffers the >> consequences of the resource/time issues for the >> matching devel/electron* having to be built. >> >> =3D=3D=3D >> Mark Millard >> marklmi at yahoo.com >> >> > =3D=3D=3D > Mark Millard > marklmi at yahoo.com > > --000000000000c0d8cd06221cd7e9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Excellent.=C2=A0 I'm glad poudriere has this chan= ge already ( https://github.com/freebsd/poudriere/commit/9= ea59e59f12f2b77699de9cafa5ceea1f7814eaf#diff-afbea33008c9478b94f3379278925c= 57204b3283411000094ee48d2388293a5f ).=C2=A0 I hadn't realized commo= n.sh had been updated and I simply needed to take out the related config fr= om my poudriere.conf=C2=A0 I'm using 3.4.1_1.

= Thanks for the assistance.


On Sat, Sep 14, 2024 at = 7:05=E2=80=AFPM Mark Millard <markl= mi@yahoo.com> wrote:
As of 2023-Dec-19 or source when FreeBSD's poudriere o= n the official builders
started being based on 3.4.0. So the MAX_FILES = default is 8192, not 4096, as
of that change. Numerous official b= uilds succeed based on the default value,
including just recently= .

If there is an explicit=C2=A0MAX_FILES_vscode=3D4096= =C2=A0somewhere that would=C2=A0use poudriere
3.4.0 or later, the assignment should simply be deleted = (instead of separately=C2=A0raised).

What version of poudriere are you using? Is it based on 3.4.0 or later?

Mark


On Sep 1= 4, 2024, at 15:45, John Schneider <john.a.schneider@gmail.com> wrote:
<= br>
Hi Mark.=C2=A0 Thanks for this very thorough = explanation.=C2=A0 I'm currently successful at building electron via po= udriere with multiple cores in a reasonable amount of time, with no special= config.=C2=A0 The thing that was biting me was the maximum number of files= knob for vscode (MAX_FILES_vscode) in poudriere.conf.=C2=A0 I imagine we&#= 39;re really close to the limit with value of 4096 at present.=C2=A0 Perhap= s sometimes 4K is enough and with VSCode source code updates we go back and= forth over this limit?

I can currently reproduce = the build error described in bug=C2=A0https://bugs.freebsd.or= g/bugzilla/show_bug.cgi?id=3D242871 a MAX_FILES_vscode=3D4096 and the b= uild is successful with a higher value.=C2=A0 I don't know if a value o= f 8192 is appropriate or not but it definitely fixes the package build.
=

Assuming we continue to have official stable pack= age builds for electron, I think the master poudriere config needs to chang= e=C2=A0 to have MAX_FILES_vscode be raised.

Thanks= ,
John

On Fri, Sep 13, 2024 at 9:12=E2=80=AFPM Mark Mill= ard <marklmi@yaho= o.com> wrote:
John Schneider <john.a.schneider_at_gmail.com> w= rote on
Date: Fri, 13 Sep 2024 21:58:52 UTC :

> I've noticed the FreeBSD package mirrors (pkg.freebsd.org) haven= 't included
> packages for editors/vscode. Reference:
> http://pkg0.nyi.freebsd.org/FreeBSD:14:a= md64/release_1/

http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/= release_1/ is
frozen as things were just before 14.1-RELEASE was made:
back on 2024-May-19. It is never updated as far as I know.
So: as-is, including what failed to be built at the time.
It is what is incuded on the official release DVD's that
have the built port packages.

http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/= release_0/ is
similar but goes back to 2023-Oct-15.

The port package repos for 14.x that are updated over time
are:

http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/= quarterly/
http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/lat= est/

In my testing just now, use of:

url: "pkg+https://pkg.FreeBSD.org/${ABI}/quarterly<= /a>",

did not find vscode as stands. Later below I report
on why/how. But use of:

url: "pkg+https://pkg.FreeBSD.org/${ABI}/latest&qu= ot;,

did find vscode.

I used a chroot into a stable/14.1 context on a
machine that booted main [so: 15] (both kernel and
world). releng/14.* and stable/14 use the same
pacakge repos via=C2=A0 the ABI text also being the same
in the URL's. . . .

# file /bin/sh
/bin/sh: ELF 64-bit LSB pie executable, x86-64, version 1 (FreeBSD), dynami= cally linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 14.1 (1401501),= FreeBSD-style, stripped

# pkg search vscode
vscode-1.92.2=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= Visual Studio Code - Open Source ("Code - OSS")

This was after a round of it synchronizing my context:

# env ABI=3DFreeBSD:14:amd64 IGNORE_OSVERSION=3Dyes pkg search vscode
pkg: Repository FreeBSD has a wrong packagesite, need to re-create database=
vscode-1.92.2=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= Visual Studio Code - Open Source ("Code - OSS")

https://www.freshports.org/editors/vscode/ reports onl= y 3 vscode
builds:

FreeBSD:13:latest has 1.92.2
FreeBSD:14:latest has 1.92.2
FreeBSD:15:latest has 1.92.2

Also, vscode is listed as depending on: devel/electron30 at
this time.

It is common for all but one devel/electron* to be
disabled for pkg builds: "blacklisted".

https://pkg-status.freebsd.org/beefy20/build.html?mastername=3D140amd= 64-quarterly&build=3D59d8804dcdd7

(started 2024-Sep-05) reports that vscode was skipped because
electron29-29.4.6 did not build at the time. Note that this
is a September example of quarterly.

In turn it reports that electron29-29.4.6 was an Ingored Port
for the Reason: "Blacklisted".

In essence the electron* 's are treated has too many using too
many resources and time to build them all without other
negative tradeoffs for the overall context.

> Could this be because the official package vscode build is failing?
I've only shown the one example. No claim that a electron*
is the only way for vscode builds to fail.

> I was
> able to overcome a problem with too many file descriptors during the b= uild
> by changing my poudriere.conf file to double MAX_FILES_vscode=3D4096 t= o
> MAX_FILES_vscode=3D8192

That will not help with the required electron* not being
available.

> This was previously addressed in bug 242871 -
> https://bugs.freebsd.org/bugzilla/sho= w_bug.cgi?id=3D242871
>
> If there's a better mailing list for this issue, please refer me t= o it.


I'll note that on the aarch64 (so: Tier 1) builder for
main (ampere2) the following all fail to build after
between 4hr and 49hr of attempting to build
(build/timeout failures):

www/ungoogled-chromium
www/iridium
www/chromium
devel/electron30

For how things are configured, attempting to build those
seems to mostly just be wasted resources and time for
aarch64. Other devel/electron* would likely be the same.
(Other configurations would have other tradeoffs and
might not prove sufficient for the overall context.)

I'll note that the next biggest time taker fails for
other reasons in under 3hr. Also, the largest time
taker that builds is www/qt6-webengine at between
27hr and 28hr currently. The next 2 builds by time
take between 19hr and 20hr currently
( databases/mongodb80 and www/qt5-webengine ).

There is a huge difference for building the various:

www/ungoogled-chromium
www/iridium
www/chromium
devel/electron*

compared to anything else. (I'm not spanning
prerequisites here: just the direct part of
the overall build for each.)

I've no clue what time limits would allow those to
all build on the ampere*, leaving things configured
the same otherwise. I also do not know what would
happen if they all started building at about the
same time, up to 13 builders, given 13 FreeBSD cpus
in each ampere* .

Anything based on a devel/electron* suffers the
consequences of the resource/time issues for the
matching devel/electron* having to be built.

=3D=3D=3D
Mark Millard
marklmi at yahoo.com


=3D=3D=3D
Mark Millard<= /font>
marklmi at yahoo.com

--000000000000c0d8cd06221cd7e9--