Re: Improving www/chromium build time on arm64

On 5/23/23 00:31, Nuno Teixeira wrote:
> Hello all,
> 
> I'm running 13-STABLE on rpi4 8 and since firefox{-esr} is constantly tab
> crashing and seg faulting I decided to give chromium a try.
> chromium pkg isn't available for arm64 and I didn't found error log at
> https://pkg-status.freebsd.org/ so I decided to build it on poudriere.
> 
> It fails to build around 70% but my biggest concern is build time: 27 hours
> for 70% compilation!
> 
> Just for comparison:
> llvm15: 12h
> rust: 9h
> firefox: 6h
> 
> rip4 v1.5 8GB @ 2000Hz ~62 Celsius
> zfs
> poudriere:
> USE_TMPFS=no
> PARALLEL_JOBS=1
> ALLOW_MAKE_JOBS=yes
> 
> Any hints on how to speed build? ccache?

I don't have that hardware, but it seems to likely be a 4 core processor.

Add:
MAKE_JOBS_NUMBER=4
to /usr/local/etc/poudriere.d/make.conf
PARALLEL_JOBS=1
to /usr/local/etc/poudriere.conf
you can also override PARALLEL_JOBS on the poudriere command line with -J.
`poudriere bulk -J1 -j local -p local www/chromium`
You now have 1 poudriere job that is allowed to run up to 4 make threads 
within its build. Other non-make scripts may not respect such limitation 
inside of it. Running more compiler commands simultaneously when 
possible will use more RAM

You can also use PREPARE_PARALLEL_JOBS=2 or -J1:2 to get 2 workers on 
other work outside of the build steps; I cannot recommend what values to 
try setting it to but you probably want higher than 1.

You can closer balance poudriere jobs vs make jobs by setting both to 2. 
Poudriere running 4 jobs with each having 1 make job means anytime there 
isn't a queue left to currently be worked on that you get more and more 
idle cores until done. Compilers, browsers,
, or maybe a mix of 3 and 2 if you don't mind oversaturating the machine 
with work.

   Using ccache would have had the first run still fail without 
adjustments but subsequent builds would seem to get farther since 
instead of taking the time to compile the code, it would get the 
compiler's answer out of the cache (presuming the answer is still there).
   I seem to have an issue where my cache does not maintain its size 
which can be resolved with `ccache -c` being occasionally ran. I tweaked 
mine to use strongest compression and a larger cache size to get more 
out of it. First runs do take slightly longer as it takes time to check 
the cache before compiling and write the result to the cache but it is 
minor for a potential large speedup later.
   I use "compiler_check = content" to try to work around the cache 
being invalidated by me rebuilding the same version of the compiler and 
reinstalling it figuring the compiler has a new timestamp then.
   I should probably disable hash_dir in ccache settings to take 
advantage of port upgrades that extract to a version dependent folder 
but have not done so; my understanding is that doing so would lead to 
debugging being more annoying as it'd want to point to files from the 
first compiled directory when the cache is used but shouldn't matter 
otherwise.

Not sure if poudriere testport would also exit on timeout but if not 
then you likely could rerun its ports tree make at the prompt until success.

   Additional minor savings may be achieved with
PKG_COMPRESSION_FORMAT= tzst
PKG_COMPRESSION_LEVEL=  best
placed in /usr/local/etc/poudriere.d/make.conf 'if' the packages of 
dependencies can now be extracted faster. Reducing 'best' would help all 
packages be created faster. The balance would require measuring time 
spent building each package vs time spent extracting it every time it is 
a dependency to figure out what format+level gives the fastest poudriere 
run time.
   Setting
USE_TMPFS="data localbase"
in /usr/local/etc/poudriere.conf would replace some disk use with RAM. 
'wrkdirs' would just force excessive swapping as chromium is commented 
to need 20GB. You could try only one or the other if RAM seems tight but 
I'd let this setting go in favor of more RAM going to compiler threads 
if forced to choose. Repeated runs that get ccache benefit need much 
less RAM for compiler cache results so you may want to disable ccache if 
trying to get measurements of used memory during a run.

> Does cflags "-O -pipe" not present in compilation is related to build time?

-O=O1=some optimizations
unspecified=-00=no optimizations
-O2=more optimizations (and present in your log)
The more optimizations you have the compiler perform, the longer the 
build will take. If you compiled dependencies of this port with better 
compiler optimizations, it 'may' decrease time needed to compile 
chromium, though I'd doubt it would be as much of a savings as the time 
was extended in each dependency.

The more build options you change, the more likely you may find obscure 
bugs in the programs that other people don't experience; even forcing 
-O0 can cause a build failure in my experience.

There are architecture specific optimizations that are accessed through 
different flags but result in less portable binaries. -march=native I 
think was the one that tells the compiler to only worry about the 
current CPU architecture for all of its used instructions and 
optimizations. Sometimes when instead you specify a particular march 
that you expect a processor to be, instructions are included that the 
processor doesn't actually support; I presume those should be reported, 
likely to llvm project directly.

-pipe is supposed to change from temporary files to pipes on some of the 
compiling steps; I haven't tested how much impact it actually has.

> Full log:
> https://people.freebsd.org/~eduardo/logs/chromium/chromium-113.0.5672.126.log
> 
> Thanks!
> 
> ---
> <SNIP>
> In file included from ../../base/check.h:11:
> ../../base/compiler_specific.h:8:10: fatal error: 'build/build_config.h'
> file not found
> #include "build/build_config.h"
>           ^~~~~~~~~~~~~~~~~~~~~~
> 1 error generated.
> ninja: build stopped: subcommand failed.
> ===> Compilation failed unexpectedly.
> Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
> the maintainer.
> *** Error code 1
> 
> Stop.
> make: stopped in /usr/ports/www/chromium
> build of www/chromium | chromium-113.0.5672.126 ended at Tue May 23
> 02:22:56 WEST 2023
> build time: 27:15:37
> !!! build failure encountered !!!
> ---