Re: Official package builder poudriere.conf update?

Reply: John Schneider : "Re: Official package builder poudriere.conf update?"
In reply to: John Schneider : "Re: Official package builder poudriere.conf update?"
Go to: [ bottom of page ] [ top of archives ] [ this month ]
From: Mark Millard <marklmi_at_yahoo.com>
Date: Sat, 14 Sep 2024 23:04:56 UTC
As of 2023-Dec-19 or source when FreeBSD's poudriere on the official builders
started being based on 3.4.0. So the MAX_FILES default is 8192, not 4096, as
of that change. Numerous official builds succeed based on the default value,
including just recently.

If there is an explicit MAX_FILES_vscode=4096 somewhere that would use poudriere
3.4.0 or later, the assignment should simply be deleted (instead of separately raised).

What version of poudriere are you using? Is it based on 3.4.0 or later?

Mark


On Sep 14, 2024, at 15:45, John Schneider <john.a.schneider@gmail.com> wrote:

Hi Mark.  Thanks for this very thorough explanation.  I'm currently successful at building electron via poudriere with multiple cores in a reasonable amount of time, with no special config.  The thing that was biting me was the maximum number of files knob for vscode (MAX_FILES_vscode) in poudriere.conf.  I imagine we're really close to the limit with value of 4096 at present.  Perhaps sometimes 4K is enough and with VSCode source code updates we go back and forth over this limit?

I can currently reproduce the build error described in bug https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=242871 a MAX_FILES_vscode=4096 and the build is successful with a higher value.  I don't know if a value of 8192 is appropriate or not but it definitely fixes the package build.

Assuming we continue to have official stable package builds for electron, I think the master poudriere config needs to change  to have MAX_FILES_vscode be raised.

Thanks,
John

On Fri, Sep 13, 2024 at 9:12 PM Mark Millard <marklmi@yahoo.com <mailto:marklmi@yahoo.com>> wrote:
> John Schneider <john.a.schneider_at_gmail.com <http://john.a.schneider_at_gmail.com/>> wrote on
> Date: Fri, 13 Sep 2024 21:58:52 UTC :
> 
> > I've noticed the FreeBSD package mirrors (pkg.freebsd.org <http://pkg.freebsd.org/>) haven't included
> > packages for editors/vscode. Reference:
> > http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/release_1/
> 
> http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/release_1/ is
> frozen as things were just before 14.1-RELEASE was made:
> back on 2024-May-19. It is never updated as far as I know.
> So: as-is, including what failed to be built at the time.
> It is what is incuded on the official release DVD's that
> have the built port packages.
> 
> http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/release_0/ is
> similar but goes back to 2023-Oct-15.
> 
> The port package repos for 14.x that are updated over time
> are:
> 
> http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/quarterly/ 
> http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/latest/ 
> 
> In my testing just now, use of:
> 
> url: "pkg+https://pkg.FreeBSD.org/${ABI}/quarterly <https://pkg.freebsd.org/$%7BABI%7D/quarterly>",
> 
> did not find vscode as stands. Later below I report
> on why/how. But use of:
> 
> url: "pkg+https://pkg.FreeBSD.org/${ABI}/latest <https://pkg.freebsd.org/$%7BABI%7D/latest>",
> 
> did find vscode.
> 
> I used a chroot into a stable/14.1 context on a
> machine that booted main [so: 15] (both kernel and
> world). releng/14.* and stable/14 use the same
> pacakge repos via  the ABI text also being the same
> in the URL's. . . .
> 
> # file /bin/sh
> /bin/sh: ELF 64-bit LSB pie executable, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 14.1 (1401501), FreeBSD-style, stripped
> 
> # pkg search vscode
> vscode-1.92.2                  Visual Studio Code - Open Source ("Code - OSS")
> 
> This was after a round of it synchronizing my context:
> 
> # env ABI=FreeBSD:14:amd64 IGNORE_OSVERSION=yes pkg search vscode
> pkg: Repository FreeBSD has a wrong packagesite, need to re-create database
> vscode-1.92.2                  Visual Studio Code - Open Source ("Code - OSS")
> 
> https://www.freshports.org/editors/vscode/ reports only 3 vscode
> builds:
> 
> FreeBSD:13:latest has 1.92.2
> FreeBSD:14:latest has 1.92.2
> FreeBSD:15:latest has 1.92.2
> 
> Also, vscode is listed as depending on: devel/electron30 at
> this time.
> 
> It is common for all but one devel/electron* to be
> disabled for pkg builds: "blacklisted".
> 
> https://pkg-status.freebsd.org/beefy20/build.html?mastername=140amd64-quarterly&build=59d8804dcdd7
> 
> (started 2024-Sep-05) reports that vscode was skipped because
> electron29-29.4.6 did not build at the time. Note that this
> is a September example of quarterly.
> 
> In turn it reports that electron29-29.4.6 was an Ingored Port
> for the Reason: "Blacklisted".
> 
> In essence the electron* 's are treated has too many using too
> many resources and time to build them all without other
> negative tradeoffs for the overall context.
> 
> > Could this be because the official package vscode build is failing?
> 
> I've only shown the one example. No claim that a electron*
> is the only way for vscode builds to fail.
> 
> > I was
> > able to overcome a problem with too many file descriptors during the build
> > by changing my poudriere.conf file to double MAX_FILES_vscode=4096 to
> > MAX_FILES_vscode=8192
> 
> That will not help with the required electron* not being
> available.
> 
> > This was previously addressed in bug 242871 -
> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=242871
> > 
> > If there's a better mailing list for this issue, please refer me to it.
> 
> 
> I'll note that on the aarch64 (so: Tier 1) builder for
> main (ampere2) the following all fail to build after
> between 4hr and 49hr of attempting to build
> (build/timeout failures):
> 
> www/ungoogled-chromium
> www/iridium
> www/chromium
> devel/electron30
> 
> For how things are configured, attempting to build those
> seems to mostly just be wasted resources and time for
> aarch64. Other devel/electron* would likely be the same.
> (Other configurations would have other tradeoffs and
> might not prove sufficient for the overall context.)
> 
> I'll note that the next biggest time taker fails for
> other reasons in under 3hr. Also, the largest time
> taker that builds is www/qt6-webengine at between
> 27hr and 28hr currently. The next 2 builds by time
> take between 19hr and 20hr currently
> ( databases/mongodb80 and www/qt5-webengine ).
> 
> There is a huge difference for building the various:
> 
> www/ungoogled-chromium
> www/iridium
> www/chromium
> devel/electron*
> 
> compared to anything else. (I'm not spanning
> prerequisites here: just the direct part of
> the overall build for each.)
> 
> I've no clue what time limits would allow those to
> all build on the ampere*, leaving things configured
> the same otherwise. I also do not know what would
> happen if they all started building at about the
> same time, up to 13 builders, given 13 FreeBSD cpus
> in each ampere* .
> 
> Anything based on a devel/electron* suffers the
> consequences of the resource/time issues for the
> matching devel/electron* having to be built.
> 
> ===
> Mark Millard
> marklmi at yahoo.com <http://yahoo.com/>
> 

===
Mark Millard
marklmi at yahoo.com