Re: Official package builder poudriere.conf update?

Excellent.  I'm glad poudriere has this change already (
https://github.com/freebsd/poudriere/commit/9ea59e59f12f2b77699de9cafa5ceea1f7814eaf#diff-afbea33008c9478b94f3379278925c57204b3283411000094ee48d2388293a5f
).  I hadn't realized common.sh had been updated and I simply needed to
take out the related config from my poudriere.conf  I'm using 3.4.1_1.

Thanks for the assistance.


On Sat, Sep 14, 2024 at 7:05 PM Mark Millard <marklmi@yahoo.com> wrote:

> As of 2023-Dec-19 or source when FreeBSD's poudriere on the official
> builders
> started being based on 3.4.0. So the MAX_FILES default is 8192, not 4096,
> as
> of that change. Numerous official builds succeed based on the default
> value,
> including just recently.
>
> If there is an explicit MAX_FILES_vscode=4096 somewhere that would use
> poudriere
> 3.4.0 or later, the assignment should simply be deleted (instead of
> separately raised).
>
> What version of poudriere are you using? Is it based on 3.4.0 or later?
>
> Mark
>
>
> On Sep 14, 2024, at 15:45, John Schneider <john.a.schneider@gmail.com>
> wrote:
>
> Hi Mark.  Thanks for this very thorough explanation.  I'm currently
> successful at building electron via poudriere with multiple cores in a
> reasonable amount of time, with no special config.  The thing that was
> biting me was the maximum number of files knob for vscode
> (MAX_FILES_vscode) in poudriere.conf.  I imagine we're really close to the
> limit with value of 4096 at present.  Perhaps sometimes 4K is enough and
> with VSCode source code updates we go back and forth over this limit?
>
> I can currently reproduce the build error described in bug
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=242871 a
> MAX_FILES_vscode=4096 and the build is successful with a higher value.  I
> don't know if a value of 8192 is appropriate or not but it definitely fixes
> the package build.
>
> Assuming we continue to have official stable package builds for electron,
> I think the master poudriere config needs to change  to have
> MAX_FILES_vscode be raised.
>
> Thanks,
> John
>
> On Fri, Sep 13, 2024 at 9:12 PM Mark Millard <marklmi@yahoo.com> wrote:
>
>> John Schneider <john.a.schneider_at_gmail.com> wrote on
>> Date: Fri, 13 Sep 2024 21:58:52 UTC :
>>
>> > I've noticed the FreeBSD package mirrors (pkg.freebsd.org) haven't
>> included
>> > packages for editors/vscode. Reference:
>> > http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/release_1/
>>
>> http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/release_1/ is
>> frozen as things were just before 14.1-RELEASE was made:
>> back on 2024-May-19. It is never updated as far as I know.
>> So: as-is, including what failed to be built at the time.
>> It is what is incuded on the official release DVD's that
>> have the built port packages.
>>
>> http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/release_0/ is
>> similar but goes back to 2023-Oct-15.
>>
>> The port package repos for 14.x that are updated over time
>> are:
>>
>> http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/quarterly/
>> http://pkg0.nyi.freebsd.org/FreeBSD:14:amd64/latest/
>>
>> In my testing just now, use of:
>>
>> url: "pkg+https://pkg.FreeBSD.org/${ABI}/quarterly
>> <https://pkg.freebsd.org/$%7BABI%7D/quarterly>",
>>
>> did not find vscode as stands. Later below I report
>> on why/how. But use of:
>>
>> url: "pkg+https://pkg.FreeBSD.org/${ABI}/latest
>> <https://pkg.freebsd.org/$%7BABI%7D/latest>",
>>
>> did find vscode.
>>
>> I used a chroot into a stable/14.1 context on a
>> machine that booted main [so: 15] (both kernel and
>> world). releng/14.* and stable/14 use the same
>> pacakge repos via  the ABI text also being the same
>> in the URL's. . . .
>>
>> # file /bin/sh
>> /bin/sh: ELF 64-bit LSB pie executable, x86-64, version 1 (FreeBSD),
>> dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 14.1
>> (1401501), FreeBSD-style, stripped
>>
>> # pkg search vscode
>> vscode-1.92.2                  Visual Studio Code - Open Source ("Code -
>> OSS")
>>
>> This was after a round of it synchronizing my context:
>>
>> # env ABI=FreeBSD:14:amd64 IGNORE_OSVERSION=yes pkg search vscode
>> pkg: Repository FreeBSD has a wrong packagesite, need to re-create
>> database
>> vscode-1.92.2                  Visual Studio Code - Open Source ("Code -
>> OSS")
>>
>> https://www.freshports.org/editors/vscode/ reports only 3 vscode
>> builds:
>>
>> FreeBSD:13:latest has 1.92.2
>> FreeBSD:14:latest has 1.92.2
>> FreeBSD:15:latest has 1.92.2
>>
>> Also, vscode is listed as depending on: devel/electron30 at
>> this time.
>>
>> It is common for all but one devel/electron* to be
>> disabled for pkg builds: "blacklisted".
>>
>>
>> https://pkg-status.freebsd.org/beefy20/build.html?mastername=140amd64-quarterly&build=59d8804dcdd7
>>
>> (started 2024-Sep-05) reports that vscode was skipped because
>> electron29-29.4.6 did not build at the time. Note that this
>> is a September example of quarterly.
>>
>> In turn it reports that electron29-29.4.6 was an Ingored Port
>> for the Reason: "Blacklisted".
>>
>> In essence the electron* 's are treated has too many using too
>> many resources and time to build them all without other
>> negative tradeoffs for the overall context.
>>
>> > Could this be because the official package vscode build is failing?
>>
>> I've only shown the one example. No claim that a electron*
>> is the only way for vscode builds to fail.
>>
>> > I was
>> > able to overcome a problem with too many file descriptors during the
>> build
>> > by changing my poudriere.conf file to double MAX_FILES_vscode=4096 to
>> > MAX_FILES_vscode=8192
>>
>> That will not help with the required electron* not being
>> available.
>>
>> > This was previously addressed in bug 242871 -
>> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=242871
>> >
>> > If there's a better mailing list for this issue, please refer me to it.
>>
>>
>> I'll note that on the aarch64 (so: Tier 1) builder for
>> main (ampere2) the following all fail to build after
>> between 4hr and 49hr of attempting to build
>> (build/timeout failures):
>>
>> www/ungoogled-chromium
>> www/iridium
>> www/chromium
>> devel/electron30
>>
>> For how things are configured, attempting to build those
>> seems to mostly just be wasted resources and time for
>> aarch64. Other devel/electron* would likely be the same.
>> (Other configurations would have other tradeoffs and
>> might not prove sufficient for the overall context.)
>>
>> I'll note that the next biggest time taker fails for
>> other reasons in under 3hr. Also, the largest time
>> taker that builds is www/qt6-webengine at between
>> 27hr and 28hr currently. The next 2 builds by time
>> take between 19hr and 20hr currently
>> ( databases/mongodb80 and www/qt5-webengine ).
>>
>> There is a huge difference for building the various:
>>
>> www/ungoogled-chromium
>> www/iridium
>> www/chromium
>> devel/electron*
>>
>> compared to anything else. (I'm not spanning
>> prerequisites here: just the direct part of
>> the overall build for each.)
>>
>> I've no clue what time limits would allow those to
>> all build on the ampere*, leaving things configured
>> the same otherwise. I also do not know what would
>> happen if they all started building at about the
>> same time, up to 13 builders, given 13 FreeBSD cpus
>> in each ampere* .
>>
>> Anything based on a devel/electron* suffers the
>> consequences of the resource/time issues for the
>> matching devel/electron* having to be built.
>>
>> ===
>> Mark Millard
>> marklmi at yahoo.com
>>
>>
> ===
> Mark Millard
> marklmi at yahoo.com
>
>