From nobody Tue May 30 10:07:21 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QVp3p35L9z4XDT7; Tue, 30 May 2023 10:07:34 +0000 (UTC) (envelope-from eduardo@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QVp3p2qvBz3wyh; Tue, 30 May 2023 10:07:34 +0000 (UTC) (envelope-from eduardo@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1685441254; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=CY43RyxpfJl52xM7QcRFjlnxMy5hmTWK0RrTJqkZOzk=; b=Dw7NBxT+YX+l2ZHbaNtwIV/4G3s4McqTwPmyRBtWuADPixC4Tlsmogfdfv0Ysd3ImZwxoJ wcgGHRAYny5VGBqm0OE06WpQjXGh6WigykbEchjqk5X4v2WSgAY2vRnRvdTH88lazJN4+n jvxjHvhevvjkAHKGtX1U+hntJerXQNpOkK6RI1Aa/Dr3mplktQvHiGdwXGOOXtIM884cSI 2ohgcy0Yirz5KSpcd8Lm72gRCZny6cFrTwTUzrbd3TydONdGbVjX3EDoY1Kk9xOotdTpMP CcLX9H/L51WxuTS25yoCgu1DuJMXPGIDcJr84Y+Kx44Zt7c+xEQBjmea/GPOpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1685441254; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=CY43RyxpfJl52xM7QcRFjlnxMy5hmTWK0RrTJqkZOzk=; b=AdVYGocy5NHRqtZ9TLsuCAp35nxcHJq10qBgOpnMxKEI7PsvJO8z0hUlbcZ1KxG/13aPTi C5EZGi83yluqJX+xbJGcjK9KlV4BsA6FzR5uQUp6+DmRigfMWBz40HPId0Ox04pmw3TDGZ ZdGIFs5sRS0DNBzGtJAYTcu+gnOSIcbSdZYb+I85DwdYbjSWZaz0KoYV8DzGS9D3rdJzyt 70orp3bOKrWjUf9KYJVzIFWqspBWdzirgDcdf9yUeNSF7nF/LRas+8OaY2z0pJd5qKfMGu eZKbJjj1atl2zaHJAgXPmqBzMFcMKHMHlu3UeGctd/4u08Ca6KQgquUr96qsug== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1685441254; a=rsa-sha256; cv=none; b=rGeipcHCXYTbP63VeumnU9gK8xwoqs1tTNfkvhpOxpL8O5oFHbfvGW5iOAe4A92BEQZxhS fpS+AX/+1W5oiISjtKe58iMKllr5ninTouzIL9AUihLIv2zEMcEgboZaWIq80YrDTb/ytf EasYdp2Gtm11hijhXmUKOjH7U+e+AHBbFHuBS3GIDTInDJREBsRYohbPfVBOp0ckVNNzwH 5rN6qYmsDdXjO7b5rW0WINW/mBhGXXT2tr9q5abf5061TBDT5K6Ls0FuXoUg6Z2aeIH0WV nNl6SiaXWEyBWELSnj9+ZhXmJM77XbBFXj2FqiG1/WouFtQIlWwqSDhKRDh8rw== Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) (Authenticated sender: eduardo) by smtp.freebsd.org (Postfix) with ESMTPSA id 4QVp3p1nPHzZw5; Tue, 30 May 2023 10:07:34 +0000 (UTC) (envelope-from eduardo@freebsd.org) Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-2564c6a2b7dso1750760a91.3; Tue, 30 May 2023 03:07:34 -0700 (PDT) X-Gm-Message-State: AC+VfDy/40JtKetZamlyRfh6jfKW2MdqzicW6D2+eSEn13Fx8YeVkjEC WN/o2V1jf+ZsMnOyHG0Lgm7z1f29xV3UaGtZzM4= X-Google-Smtp-Source: ACHHUZ76ZZH35M1Y0e6eFcPjRgLcUNYd+4QZX7N+mr/WQ29kpBCbS3slkT75UpsxAT+hVLbqa9donJ06GTtue9phS9s= X-Received: by 2002:a17:90a:cb93:b0:256:6c75:b512 with SMTP id a19-20020a17090acb9300b002566c75b512mr2101901pju.46.1685441253048; Tue, 30 May 2023 03:07:33 -0700 (PDT) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 References: <1623315797.5.1684837400228@mailrelay> In-Reply-To: From: Nuno Teixeira Date: Tue, 30 May 2023 11:07:21 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Improving www/chromium build time on arm64 To: Mark Millard Cc: Ronald Klop , FreeBSD Mailing List , freebsd-arm@freebsd.org Content-Type: multipart/alternative; boundary="0000000000002d87fc05fce65f39" X-ThisMailContainsUnwantedMimeParts: N --0000000000002d87fc05fce65f39 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello Mark, Wrong poudriere.conf. Uploaded correct one. Summary: --- USE_TMPFS=3Dno PARALLEL_JOBS=3D1 ALLOW_MAKE_JOBS=3Dyes MAX_EXECUTION_TIME=3D259200 NOHANG_TIME=3D259200 --- htop showed me 4 escreveu no dia ter=C3=A7a, 30/05/2023 =C3= =A0(s) 10:47: > On May 30, 2023, at 01:39, Nuno Teixeira wrote: > > > Hello Ronald and all! > > > > Finally I build it after 57 hours @ 2000MHz !!! > > So the prior "[88:53:28] Failed ports: www/chromium:build/timeout" > was not with the overclocking? Other differences that could > contribute to the time difference? > > MAX_EXECUTION_TIME=3D86400 (the default) only allows for 24 hours > before it starts analyzing the log for failure information (while > letting the build continue while that scanning is done). > > > Configs, photos and log at: > https://people.freebsd.org/~eduardo/logs/chromium/ > > https://people.freebsd.org/~eduardo/logs/chromium/poudriere.conf > shows no assignments to MAX_EXECUTION_TIME or NOHANG_TIME or > other such (outside comments). In the comments are the default > values, not increases. > > Similarly for PARALLEL_JOBS , PREPARE_PARALLEL_JOBS , ALLOW_MAKE_JOBS , > and so on. (It is not clear what the load average behavior/ idle time > behavior might have been.) > > The file does show explicit assignments to: ZPOOL, FREEBSD_HOST , > RESOLV_CONF , BASEFS , USE_PORTLINT , USE_TMPFS , DISTFILES_CACHE , > and PACKAGE_FETCH_URL . > > > I'm running it right now and it runs very smooth! > > > > Next step will be compiling with LTO to check compile time and do some > comparisons on binary size and run performance. > > > > Should I bother with LTO? Will it add performance? > > > > Thanks all! > > > > > > Ronald Klop escreveu no dia ter=C3=A7a, 23/05/20= 23 > =C3=A0(s) 11:23: > >> =C3=82 > >>> Van: Nuno Teixeira > >>> Datum: dinsdag, 23 mei 2023 09:48 > >>> Aan: FreeBSD Mailing List , > freebsd-arm@freebsd.org > >>> Onderwerp: Re: Improving www/chromium build time on arm64 > >>> (...) > >>> =C3=82 > >>> But I also found that poudriere: > >>> [88:53:28] Failed ports: www/chromium:build/timeout > >>> =C3=82 > >>> So I will search how do I increase build time to avoid timeout: > >>> --- > >>> # This defines the max time (in seconds) that a command may run for a > build > >>> # before it is killed for taking too long. Default: 86400 > >>> #MAX_EXECUTION_TIME=3D86400 > >>> --- > >>> =C3=82 > >>> I suspect it can be the cause of build failure because build >24h. > >>> =C3=82 > >>> Nuno Teixeira escreveu no dia ter=C3=83=C2=A7a, > 23/05/2023 =C3=83 (s) 08:31: > >>> Hello all, > >>> =C3=82 > >>> I'm running 13-STABLE on rpi4 8 and since firefox{-esr} is constantly > tab crashing and seg faulting I decided to give chromium a try. > >>> chromium pkg isn't available for arm64 and I didn't found error log a= t > https://pkg-status.freebsd.org/ so I decided to build it on poudriere. > >>> =C3=82 > >>> It fails to build around 70% but my biggest concern is build time: 27 > hours for 70% compilation! > >>> =C3=82 > >>> Just for comparison: > >>> llvm15: 12h > >>> rust: 9h > >>> firefox: 6h > >>> =C3=82 > >>> rip4 v1.5 8GB @ 2000Hz ~62 Celsius > >>> zfs > >>> poudriere: > >>> USE_TMPFS=3Dno > >>> PARALLEL_JOBS=3D1 > >>> ALLOW_MAKE_JOBS=3Dyes > >>> =C3=82 > >>> Any hints on how to speed build? ccache? > >>> Does cflags "-O -pipe" not present in compilation is related to build > time? > >>> =C3=82 > >>> Full log: > >>> > https://people.freebsd.org/~eduardo/logs/chromium/chromium-113.0.5672.126= .log > >>> =C3=82 > >>> Thanks! > >>> =C3=82 > >>> --- > >>> > >>> In file included from ../../base/check.h:11: > >>> ../../base/compiler_specific.h:8:10: fatal error: > 'build/build_config.h' file not found > >>> #include "build/build_config.h" > >>> ^~~~~~~~~~~~~~~~~~~~~~ > >>> 1 error generated. > >>> ninja: build stopped: subcommand failed. > >>> =3D=3D=3D> Compilation failed unexpectedly. > >>> Try to set MAKE_JOBS_UNSAFE=3Dyes and rebuild before reporting the > failure to > >>> the maintainer. > >>> *** Error code 1 > >>> > >>> Stop. > >>> make: stopped in /usr/ports/www/chromium > >>> build of www/chromium | chromium-113.0.5672.126 ended at Tue May 23 > 02:22:56 WEST 2023 > >>> build time: 27:15:37 > >>> !!! build failure encountered !!! > >>> --- > >>> -- Nuno Teixeira > >>> FreeBSD Committer (ports) > >>> =C3=82 > >>> -- Nuno Teixeira > >>> FreeBSD Committer (ports) > >>> > >> > >> Hi, > >> > >> My poudriere.conf for building ports on rpi4 contains these two change= s: > >> > >> # This defines the max time (in seconds) that a command may run for a > build > >> # before it is killed for taking too long. Default: 86400 > >> #MAX_EXECUTION_TIME=3D86400 > >> MAX_EXECUTION_TIME=3D172800 > >> > >> # This defines the time (in seconds) before a command is considered to > >> # be in a runaway state for having no output on stdout. Default: 7200 > >> #NOHANG_TIME=3D7200 > >> NOHANG_TIME=3D172800 > >> > >> > >> I'm maintaining some mongodb ports which also take a long long time to > build on rpi4. > >> But I don't remember which of these settings has the preferred impact > on your situation. > >> > >> Oh... now I'm reading your log better. It contains this: > >> "=3D>> Killing timed out build after 86400 seconds" > >> > >> And some time later: > >> "=3D>> Cleaning up wrkdir > >> =3D=3D=3D> Cleaning for chromium-113.0.5672.126" > >> > >> So the errors about missing files happen because poudriere is removing > files while some processes of the build are still running. > >> > >> I'm curious how far it gets after you increase MAX_EXECUTION_TIME. > > > > =3D=3D=3D > Mark Millard > marklmi at yahoo.com > > --=20 Nuno Teixeira FreeBSD Committer (ports) --0000000000002d87fc05fce65f39 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello Mark,

Wrong poudriere.= conf. Uploaded correct one.

Summary:
---=
USE_TMPFS=3Dno
PARALLEL_JOBS=3D1
ALLOW_MAKE_JOBS=3D= yes
MAX_EXECUTION_TIME=3D259200
NOHANG_TIME=3D259200
---

htop showed me 4<load<5 (around 4,= 70) with temperature at ~60/65 degrees Celsius

Mark Millard <= marklmi@yahoo.com> escreveu no = dia ter=C3=A7a, 30/05/2023 =C3=A0(s) 10:47:
On May 30, 2023, at 01:39, Nuno Teixeira <eduardo@freebsd.org<= /a>> wrote:

> Hello Ronald and all!
>
> Finally I build it after 57 hours @ 2000MHz !!!

So the prior "[88:53:28] Failed ports: www/chromium:build/timeout"= ;
was not with the overclocking? Other differences that could
contribute to the time difference?

MAX_EXECUTION_TIME=3D86400 (the default) only allows for 24 hours
before it starts analyzing the log for failure information (while
letting the build continue while that scanning is done).

> Configs, photos and log at: https://people.fr= eebsd.org/~eduardo/logs/chromium/

https://people.freebsd.org/~eduardo/= logs/chromium/poudriere.conf
shows no assignments to MAX_EXECUTION_TIME or NOHANG_TIME or
other such (outside comments). In the comments are the default
values, not increases.

Similarly for PARALLEL_JOBS , PREPARE_PARALLEL_JOBS , ALLOW_MAKE_JOBS ,
and so on. (It is not clear what the load average behavior/ idle time
behavior might have been.)

The file does show explicit assignments to: ZPOOL, FREEBSD_HOST ,
RESOLV_CONF , BASEFS , USE_PORTLINT , USE_TMPFS , DISTFILES_CACHE ,
and PACKAGE_FETCH_URL .

> I'm running it right now and it runs very smooth!
>
> Next step will be compiling with LTO to check compile time and do some= comparisons on binary size and run performance.
>
> Should I bother with LTO? Will it add performance?
>
> Thanks all!
>
>
> Ronald Klop <ronald-lists@klop.ws> escreveu no dia ter=C3=A7a, 23/05/2023 =C3= =A0(s) 11:23:
>> =C3=82=C2=A0
>>> Van: Nuno Teixeira <eduardo@freebsd.org>
>>> Datum: dinsdag, 23 mei 2023 09:48
>>> Aan: FreeBSD Mailing List <freebsd-ports@freebsd.org>, freebsd-arm@freebsd.o= rg
>>> Onderwerp: Re: Improving www/chromium build time on arm64
>>> (...)
>>> =C3=82
>>> But I also found that poudriere:
>>> [88:53:28] Failed ports: www/chromium:build/timeout
>>> =C3=82
>>> So I will search how do I increase build time to avoid timeout= :
>>> ---
>>> # This defines the max time (in seconds) that a command may ru= n for a build
>>> # before it is killed for taking too long. Default: 86400
>>> #MAX_EXECUTION_TIME=3D86400
>>> ---
>>> =C3=82
>>> I suspect it can be the cause of build failure because build &= gt;24h.
>>> =C3=82=C2=A0
>>> Nuno Teixeira <eduardo@freebsd.org> escreveu no dia ter=C3=83=C2=A7a, = 23/05/2023 =C3=83 (s) 08:31:
>>> Hello all,
>>> =C3=82
>>> I'm running 13-STABLE on rpi4 8 and since firefox{-esr} is= constantly tab crashing and seg faulting I decided to give chromium a try.=
>>> chromium pkg isn't available for arm64 and I didn't fo= und error log at https://pkg-status.freebsd.org/ so I decided to = build it on poudriere.
>>> =C3=82
>>> It fails to build around 70% but my biggest concern is build t= ime: 27 hours for 70% compilation!
>>> =C3=82
>>> Just for comparison:
>>> llvm15: 12h
>>> rust: 9h
>>> firefox: 6h
>>> =C3=82
>>> rip4 v1.5 8GB @ 2000Hz ~62 Celsius
>>> zfs
>>> poudriere:
>>> USE_TMPFS=3Dno
>>> PARALLEL_JOBS=3D1
>>> ALLOW_MAKE_JOBS=3Dyes
>>> =C3=82
>>> Any hints on how to speed build? ccache?
>>> Does cflags "-O -pipe" not present in compilation is= related to build time?
>>> =C3=82
>>> Full log:
>>> https://pe= ople.freebsd.org/~eduardo/logs/chromium/chromium-113.0.5672.126.log
>>> =C3=82
>>> Thanks!
>>> =C3=82
>>> ---
>>> <SNIP>
>>> In file included from ../../base/check.h:11:
>>> ../../base/compiler_specific.h:8:10: fatal error: 'build/b= uild_config.h' file not found
>>> #include "build/build_config.h"
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ^~~~~~~~~~~~~~~~~~~~~~
>>> 1 error generated.
>>> ninja: build stopped: subcommand failed.
>>> =3D=3D=3D> Compilation failed unexpectedly.
>>> Try to set MAKE_JOBS_UNSAFE=3Dyes and rebuild before reporting= the failure to
>>> the maintainer.
>>> *** Error code 1
>>>
>>> Stop.
>>> make: stopped in /usr/ports/www/chromium
>>> build of www/chromium | chromium-113.0.5672.126 ended at Tue M= ay 23 02:22:56 WEST 2023
>>> build time: 27:15:37
>>> !!! build failure encountered !!!
>>> ---
>>> -- Nuno Teixeira
>>> FreeBSD Committer (ports)
>>>=C2=A0 =C2=A0=C3=82
>>> -- Nuno Teixeira
>>> FreeBSD Committer (ports)
>>>
>>
>> Hi,
>>
>> My poudriere.conf for building ports on rpi4 contains these two ch= anges:
>>
>> # This defines the max time (in seconds) that a command may run fo= r a build
>> # before it is killed for taking too long. Default: 86400
>> #MAX_EXECUTION_TIME=3D86400
>> MAX_EXECUTION_TIME=3D172800
>>
>> # This defines the time (in seconds) before a command is considere= d to
>> # be in a runaway state for having no output on stdout. Default: 7= 200
>> #NOHANG_TIME=3D7200
>> NOHANG_TIME=3D172800
>>
>>
>> I'm maintaining some mongodb ports which also take a long long= time to build on rpi4.
>> But I don't remember which of these settings has the preferred= impact on your situation.
>>
>> Oh... now I'm reading your log better. It contains this:
>> "=3D>> Killing timed out build after 86400 seconds"= ;
>>
>> And some time later:
>> "=3D>> Cleaning up wrkdir
>> =3D=3D=3D> Cleaning for chromium-113.0.5672.126"
>>
>> So the errors about missing files happen because poudriere is remo= ving files while some processes of the build are still running.
>>
>> I'm curious how far it gets after you increase MAX_EXECUTION_T= IME.
>

=3D=3D=3D
Mark Millard
marklmi at yahoo.com



--
Nuno Teixeira
FreeBSD Committ= er (ports)
--0000000000002d87fc05fce65f39--