From nobody Tue May 30 10:53:24 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QVq4w5rqXz4XHm5; Tue, 30 May 2023 10:53:36 +0000 (UTC) (envelope-from eduardo@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QVq4w5Z9Sz42Fl; Tue, 30 May 2023 10:53:36 +0000 (UTC) (envelope-from eduardo@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1685444016; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=OMGZOfJxdCklPyOgYNt9CKatorQsvCs251XH9jbF6aQ=; b=UuuCQpeETLUEdGpj0onIDhvvdDAdraWw+uQDLF2d65xgQDdmYp7sYsBKlHEK/N4WRDciAS 2suDQ5xezIkcSy8CSwkaSpGmHbUIBZqaNrWUIQnFd9sRyJNQZzDTt4P9a/lae47bc6TGqh MOi+428KnPSG8Y+TVobiVR+EQIbOcyk9veEjUGOVL2wZBUrRg0rab9G+9cwx20PSN4UGD3 9fxG3pNs0pqFX702l2q6bgEi1UZzXihGkmXOCLrYgavj9r8+fIYcc2igmz7kACtAUQAxN0 9b2Klf5onxfMzJtbIH0vjC3JJgHx29ibSrswG/mwXZvnoG0sxlf6QWfCu7MHag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1685444016; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=OMGZOfJxdCklPyOgYNt9CKatorQsvCs251XH9jbF6aQ=; b=cuJulQfN4G9zpGURs6rG8bZIza6FUZQRjefoKVPZxeMy0ylTCzjWokd8Ub4sBBZJSG/Hn+ DQSiUrRv/iRYXNODhc4H0nBLqISrOIe/tqCL5Us4Ci2oMFe2vbq9ifZUBccE/crNXCKs7c mWqFTIsRfrJwJClkhTT7Q5ayX9mQbsBvOWS6wSPLmwbYO1oiGFMHVr4+yyo/OEr53y/pN5 KQ8KxNStUvo60K2Gx6JgNP2oAHwjmFZs8ptByXOTXpcxJteUlltVJ2kfN6R4dRcYUzR4WT Ajd0nxv26eXyY7Y47SKw0GiZKe/j6gQNt1xhMlnIKKG720KHkICXKqXPuAJTZw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1685444016; a=rsa-sha256; cv=none; b=XN8mVGbe4P1fthQtCDqeP2GiDAV2Nuw5t/MjSjZCMF45uG8YqlWQL+XrfeOfK0KXZlGe7R V6NEeD9LTC0SJDjX1d+ygWJQo+9g6pxyS16hA1ouryK7kKFeBvHshOp4hF5Idlmw/UjSsp oR1m2cftO0BgqxBdQ7hW4TA0TS8NIC6SjpqHVWzOlfg/81za36AH5ZMCUyJvHc6z5SON3M KGm93NQho61rXU7Zhbsi7BtZhr9XvRi7j9jVL/7pB9+4ImrepR0I8tQ9YM/IDmaKZBTiz3 Z9SUcgN+oa2fAzqnYxv58AZ0sWLqeQG5tFaeRbPuulQ/Gpba7VOk113ifacKrg== Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) (Authenticated sender: eduardo) by smtp.freebsd.org (Postfix) with ESMTPSA id 4QVq4w4FL7zbNl; Tue, 30 May 2023 10:53:36 +0000 (UTC) (envelope-from eduardo@freebsd.org) Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-64d2a87b9daso3135651b3a.0; Tue, 30 May 2023 03:53:36 -0700 (PDT) X-Gm-Message-State: AC+VfDyjLjhHCgFTQ021laE06E+goZYAFNPTzvCzGBlEZz7PJ1f/wLY1 IEr3uSwNdlQZdbjyWpp1oJhKmOROMI+mVmwTCyA= X-Google-Smtp-Source: ACHHUZ540IPg68rmeNuSrNDgeOmgwV9wm7BK5B86Brb2srE1lZ2bVfRiYvzO9ggoPdn8JY748AxlyInT+4xaF3fkEIo= X-Received: by 2002:a05:6a20:a124:b0:10b:2214:7ab1 with SMTP id q36-20020a056a20a12400b0010b22147ab1mr2433320pzk.18.1685444015314; Tue, 30 May 2023 03:53:35 -0700 (PDT) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 References: <1623315797.5.1684837400228@mailrelay> In-Reply-To: From: Nuno Teixeira Date: Tue, 30 May 2023 11:53:24 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Improving www/chromium build time on arm64 To: Mark Millard Cc: Ronald Klop , FreeBSD Mailing List , freebsd-arm@freebsd.org Content-Type: multipart/alternative; boundary="000000000000d25d3f05fce70368" X-ThisMailContainsUnwantedMimeParts: N --000000000000d25d3f05fce70368 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable (...) (I've missed OPTIONS_EXCLUDE_aarch64=3DLTO so forgeting it.) Conlusion: Thats a big difference on compilation times: --- build of www/firefox | firefox-113.0.2_1,2 build time: 05:35:57 --- build of www/chromium | chromium-113.0.5672.126 build time: 56:59:23 --- But I'm enjoying the fast and smooth chrome run. Really nice. Nuno Teixeira escreveu no dia ter=C3=A7a, 30/05/2023 = =C3=A0(s) 11:07: > Hello Mark, > > Wrong poudriere.conf. Uploaded correct one. > > Summary: > --- > USE_TMPFS=3Dno > PARALLEL_JOBS=3D1 > ALLOW_MAKE_JOBS=3Dyes > MAX_EXECUTION_TIME=3D259200 > NOHANG_TIME=3D259200 > --- > > htop showed me 4 Celsius > > Mark Millard escreveu no dia ter=C3=A7a, 30/05/2023 = =C3=A0(s) > 10:47: > >> On May 30, 2023, at 01:39, Nuno Teixeira wrote: >> >> > Hello Ronald and all! >> > >> > Finally I build it after 57 hours @ 2000MHz !!! >> >> So the prior "[88:53:28] Failed ports: www/chromium:build/timeout" >> was not with the overclocking? Other differences that could >> contribute to the time difference? >> >> MAX_EXECUTION_TIME=3D86400 (the default) only allows for 24 hours >> before it starts analyzing the log for failure information (while >> letting the build continue while that scanning is done). >> >> > Configs, photos and log at: >> https://people.freebsd.org/~eduardo/logs/chromium/ >> >> https://people.freebsd.org/~eduardo/logs/chromium/poudriere.conf >> shows no assignments to MAX_EXECUTION_TIME or NOHANG_TIME or >> other such (outside comments). In the comments are the default >> values, not increases. >> >> Similarly for PARALLEL_JOBS , PREPARE_PARALLEL_JOBS , ALLOW_MAKE_JOBS , >> and so on. (It is not clear what the load average behavior/ idle time >> behavior might have been.) >> >> The file does show explicit assignments to: ZPOOL, FREEBSD_HOST , >> RESOLV_CONF , BASEFS , USE_PORTLINT , USE_TMPFS , DISTFILES_CACHE , >> and PACKAGE_FETCH_URL . >> >> > I'm running it right now and it runs very smooth! >> > >> > Next step will be compiling with LTO to check compile time and do some >> comparisons on binary size and run performance. >> > >> > Should I bother with LTO? Will it add performance? >> > >> > Thanks all! >> > >> > >> > Ronald Klop escreveu no dia ter=C3=A7a, 23/05/2= 023 >> =C3=A0(s) 11:23: >> >> =C3=82 >> >>> Van: Nuno Teixeira >> >>> Datum: dinsdag, 23 mei 2023 09:48 >> >>> Aan: FreeBSD Mailing List , >> freebsd-arm@freebsd.org >> >>> Onderwerp: Re: Improving www/chromium build time on arm64 >> >>> (...) >> >>> =C3=82 >> >>> But I also found that poudriere: >> >>> [88:53:28] Failed ports: www/chromium:build/timeout >> >>> =C3=82 >> >>> So I will search how do I increase build time to avoid timeout: >> >>> --- >> >>> # This defines the max time (in seconds) that a command may run for = a >> build >> >>> # before it is killed for taking too long. Default: 86400 >> >>> #MAX_EXECUTION_TIME=3D86400 >> >>> --- >> >>> =C3=82 >> >>> I suspect it can be the cause of build failure because build >24h. >> >>> =C3=82 >> >>> Nuno Teixeira escreveu no dia ter=C3=83=C2=A7a= , >> 23/05/2023 =C3=83 (s) 08:31: >> >>> Hello all, >> >>> =C3=82 >> >>> I'm running 13-STABLE on rpi4 8 and since firefox{-esr} is constantl= y >> tab crashing and seg faulting I decided to give chromium a try. >> >>> chromium pkg isn't available for arm64 and I didn't found error log >> at https://pkg-status.freebsd.org/ so I decided to build it on poudriere= . >> >>> =C3=82 >> >>> It fails to build around 70% but my biggest concern is build time: 2= 7 >> hours for 70% compilation! >> >>> =C3=82 >> >>> Just for comparison: >> >>> llvm15: 12h >> >>> rust: 9h >> >>> firefox: 6h >> >>> =C3=82 >> >>> rip4 v1.5 8GB @ 2000Hz ~62 Celsius >> >>> zfs >> >>> poudriere: >> >>> USE_TMPFS=3Dno >> >>> PARALLEL_JOBS=3D1 >> >>> ALLOW_MAKE_JOBS=3Dyes >> >>> =C3=82 >> >>> Any hints on how to speed build? ccache? >> >>> Does cflags "-O -pipe" not present in compilation is related to buil= d >> time? >> >>> =C3=82 >> >>> Full log: >> >>> >> https://people.freebsd.org/~eduardo/logs/chromium/chromium-113.0.5672.12= 6.log >> >>> =C3=82 >> >>> Thanks! >> >>> =C3=82 >> >>> --- >> >>> >> >>> In file included from ../../base/check.h:11: >> >>> ../../base/compiler_specific.h:8:10: fatal error: >> 'build/build_config.h' file not found >> >>> #include "build/build_config.h" >> >>> ^~~~~~~~~~~~~~~~~~~~~~ >> >>> 1 error generated. >> >>> ninja: build stopped: subcommand failed. >> >>> =3D=3D=3D> Compilation failed unexpectedly. >> >>> Try to set MAKE_JOBS_UNSAFE=3Dyes and rebuild before reporting the >> failure to >> >>> the maintainer. >> >>> *** Error code 1 >> >>> >> >>> Stop. >> >>> make: stopped in /usr/ports/www/chromium >> >>> build of www/chromium | chromium-113.0.5672.126 ended at Tue May 23 >> 02:22:56 WEST 2023 >> >>> build time: 27:15:37 >> >>> !!! build failure encountered !!! >> >>> --- >> >>> -- Nuno Teixeira >> >>> FreeBSD Committer (ports) >> >>> =C3=82 >> >>> -- Nuno Teixeira >> >>> FreeBSD Committer (ports) >> >>> >> >> >> >> Hi, >> >> >> >> My poudriere.conf for building ports on rpi4 contains these two >> changes: >> >> >> >> # This defines the max time (in seconds) that a command may run for a >> build >> >> # before it is killed for taking too long. Default: 86400 >> >> #MAX_EXECUTION_TIME=3D86400 >> >> MAX_EXECUTION_TIME=3D172800 >> >> >> >> # This defines the time (in seconds) before a command is considered t= o >> >> # be in a runaway state for having no output on stdout. Default: 7200 >> >> #NOHANG_TIME=3D7200 >> >> NOHANG_TIME=3D172800 >> >> >> >> >> >> I'm maintaining some mongodb ports which also take a long long time t= o >> build on rpi4. >> >> But I don't remember which of these settings has the preferred impact >> on your situation. >> >> >> >> Oh... now I'm reading your log better. It contains this: >> >> "=3D>> Killing timed out build after 86400 seconds" >> >> >> >> And some time later: >> >> "=3D>> Cleaning up wrkdir >> >> =3D=3D=3D> Cleaning for chromium-113.0.5672.126" >> >> >> >> So the errors about missing files happen because poudriere is removin= g >> files while some processes of the build are still running. >> >> >> >> I'm curious how far it gets after you increase MAX_EXECUTION_TIME. >> > >> >> =3D=3D=3D >> Mark Millard >> marklmi at yahoo.com >> >> > > -- > Nuno Teixeira > FreeBSD Committer (ports) > --=20 Nuno Teixeira FreeBSD Committer (ports) --000000000000d25d3f05fce70368 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
(...)

(I've missed OPTIO= NS_EXCLUDE_aarch64=3DLTO so forgeting it.)

Con= lusion:

Thats a big difference on compilation time= s:

---
build of www/firefox | firefo= x-113.0.2_1,2
build time: 05:35:57
---
build of = www/chromium | chromium-113.0.5672.126
build time: 56:59:23
---
But I'm enjoying the fast and smooth chrome = run.
Really nice.

<= div dir=3D"ltr" class=3D"gmail_attr">Nuno Teixeira <eduardo@freebsd.org> escreveu no dia ter=C3=A7a, = 30/05/2023 =C3=A0(s) 11:07:
Hello Mark,

Wron= g poudriere.conf. Uploaded correct one.

Summary:
---
USE_TMPFS=3Dno
PARALLEL_JOBS=3D1
ALLOW_= MAKE_JOBS=3Dyes
MAX_EXECUTION_TIME=3D259200
NOHANG_TIME= =3D259200
---

htop showed me 4<load&l= t;5 (around 4,70) with temperature at ~60/65 degrees Celsius

Mar= k Millard <marklm= i@yahoo.com> escreveu no dia ter=C3=A7a, 30/05/2023 =C3=A0(s) 10:47:=
On May 30, 2023= , at 01:39, Nuno Teixeira <eduardo@freebsd.org> wrote:

> Hello Ronald and all!
>
> Finally I build it after 57 hours @ 2000MHz !!!

So the prior "[88:53:28] Failed ports: www/chromium:build/timeout"= ;
was not with the overclocking? Other differences that could
contribute to the time difference?

MAX_EXECUTION_TIME=3D86400 (the default) only allows for 24 hours
before it starts analyzing the log for failure information (while
letting the build continue while that scanning is done).

> Configs, photos and log at: https://people.fr= eebsd.org/~eduardo/logs/chromium/

https://people.freebsd.org/~eduardo/= logs/chromium/poudriere.conf
shows no assignments to MAX_EXECUTION_TIME or NOHANG_TIME or
other such (outside comments). In the comments are the default
values, not increases.

Similarly for PARALLEL_JOBS , PREPARE_PARALLEL_JOBS , ALLOW_MAKE_JOBS ,
and so on. (It is not clear what the load average behavior/ idle time
behavior might have been.)

The file does show explicit assignments to: ZPOOL, FREEBSD_HOST ,
RESOLV_CONF , BASEFS , USE_PORTLINT , USE_TMPFS , DISTFILES_CACHE ,
and PACKAGE_FETCH_URL .

> I'm running it right now and it runs very smooth!
>
> Next step will be compiling with LTO to check compile time and do some= comparisons on binary size and run performance.
>
> Should I bother with LTO? Will it add performance?
>
> Thanks all!
>
>
> Ronald Klop <ronald-lists@klop.ws> escreveu no dia ter=C3=A7a, 23/05/2023 =C3= =A0(s) 11:23:
>> =C3=82=C2=A0
>>> Van: Nuno Teixeira <eduardo@freebsd.org>
>>> Datum: dinsdag, 23 mei 2023 09:48
>>> Aan: FreeBSD Mailing List <freebsd-ports@freebsd.org>, freebsd-arm@freebsd.o= rg
>>> Onderwerp: Re: Improving www/chromium build time on arm64
>>> (...)
>>> =C3=82
>>> But I also found that poudriere:
>>> [88:53:28] Failed ports: www/chromium:build/timeout
>>> =C3=82
>>> So I will search how do I increase build time to avoid timeout= :
>>> ---
>>> # This defines the max time (in seconds) that a command may ru= n for a build
>>> # before it is killed for taking too long. Default: 86400
>>> #MAX_EXECUTION_TIME=3D86400
>>> ---
>>> =C3=82
>>> I suspect it can be the cause of build failure because build &= gt;24h.
>>> =C3=82=C2=A0
>>> Nuno Teixeira <eduardo@freebsd.org> escreveu no dia ter=C3=83=C2=A7a, = 23/05/2023 =C3=83 (s) 08:31:
>>> Hello all,
>>> =C3=82
>>> I'm running 13-STABLE on rpi4 8 and since firefox{-esr} is= constantly tab crashing and seg faulting I decided to give chromium a try.=
>>> chromium pkg isn't available for arm64 and I didn't fo= und error log at https://pkg-status.freebsd.org/ so I decided to = build it on poudriere.
>>> =C3=82
>>> It fails to build around 70% but my biggest concern is build t= ime: 27 hours for 70% compilation!
>>> =C3=82
>>> Just for comparison:
>>> llvm15: 12h
>>> rust: 9h
>>> firefox: 6h
>>> =C3=82
>>> rip4 v1.5 8GB @ 2000Hz ~62 Celsius
>>> zfs
>>> poudriere:
>>> USE_TMPFS=3Dno
>>> PARALLEL_JOBS=3D1
>>> ALLOW_MAKE_JOBS=3Dyes
>>> =C3=82
>>> Any hints on how to speed build? ccache?
>>> Does cflags "-O -pipe" not present in compilation is= related to build time?
>>> =C3=82
>>> Full log:
>>> https://pe= ople.freebsd.org/~eduardo/logs/chromium/chromium-113.0.5672.126.log
>>> =C3=82
>>> Thanks!
>>> =C3=82
>>> ---
>>> <SNIP>
>>> In file included from ../../base/check.h:11:
>>> ../../base/compiler_specific.h:8:10: fatal error: 'build/b= uild_config.h' file not found
>>> #include "build/build_config.h"
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ^~~~~~~~~~~~~~~~~~~~~~
>>> 1 error generated.
>>> ninja: build stopped: subcommand failed.
>>> =3D=3D=3D> Compilation failed unexpectedly.
>>> Try to set MAKE_JOBS_UNSAFE=3Dyes and rebuild before reporting= the failure to
>>> the maintainer.
>>> *** Error code 1
>>>
>>> Stop.
>>> make: stopped in /usr/ports/www/chromium
>>> build of www/chromium | chromium-113.0.5672.126 ended at Tue M= ay 23 02:22:56 WEST 2023
>>> build time: 27:15:37
>>> !!! build failure encountered !!!
>>> ---
>>> -- Nuno Teixeira
>>> FreeBSD Committer (ports)
>>>=C2=A0 =C2=A0=C3=82
>>> -- Nuno Teixeira
>>> FreeBSD Committer (ports)
>>>
>>
>> Hi,
>>
>> My poudriere.conf for building ports on rpi4 contains these two ch= anges:
>>
>> # This defines the max time (in seconds) that a command may run fo= r a build
>> # before it is killed for taking too long. Default: 86400
>> #MAX_EXECUTION_TIME=3D86400
>> MAX_EXECUTION_TIME=3D172800
>>
>> # This defines the time (in seconds) before a command is considere= d to
>> # be in a runaway state for having no output on stdout. Default: 7= 200
>> #NOHANG_TIME=3D7200
>> NOHANG_TIME=3D172800
>>
>>
>> I'm maintaining some mongodb ports which also take a long long= time to build on rpi4.
>> But I don't remember which of these settings has the preferred= impact on your situation.
>>
>> Oh... now I'm reading your log better. It contains this:
>> "=3D>> Killing timed out build after 86400 seconds"= ;
>>
>> And some time later:
>> "=3D>> Cleaning up wrkdir
>> =3D=3D=3D> Cleaning for chromium-113.0.5672.126"
>>
>> So the errors about missing files happen because poudriere is remo= ving files while some processes of the build are still running.
>>
>> I'm curious how far it gets after you increase MAX_EXECUTION_T= IME.
>

=3D=3D=3D
Mark Millard
marklmi at yahoo.com



--
Nuno Teixeira
FreeBSD Committ= er (ports)


--
Nuno Teixeira
FreeBSD Committ= er (ports)
--000000000000d25d3f05fce70368--