From nobody Fri Mar 15 08:59:45 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Twyqy4h6Zz5CyGf; Fri, 15 Mar 2024 08:59:58 +0000 (UTC) (envelope-from oliver.epper@gmail.com) Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Twyqy065zz4r85; Fri, 15 Mar 2024 08:59:58 +0000 (UTC) (envelope-from oliver.epper@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pg1-x52b.google.com with SMTP id 41be03b00d2f7-5e42b4bbfa4so1211600a12.1; Fri, 15 Mar 2024 01:59:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1710493196; x=1711097996; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=8OmIAT24hWmP9ZRPzXqyUUK9u26w0CUFTu8E1G/NvBg=; b=JPazQ9yxFwi87CRnrp+GLCGMxdf8pDjhZJ0anlTufFf3D8tNooOmRLC9mBv7hmY8WC WGAKVBAVkAlpm9a+0/aqKJxEcDiAh/qRHK/HKccwJms4dNRqpNH7Us6ln1kkaGM9x4yN wTRO9S3s6EUAprx0B3jzbzZKnoLD9XFy5ONSzQQdXZ+fFkXl4Jl1av0/UzrdDzWyn05n p2lzMyP3Ls+36mI3JLbJADM+SjbE2VGOSUyPaVdYvbGv41REiEXktz4+zeI39Fb8H/dF W377M5jVfeJ22gpUOxk42ikphN5m8WuNPqCLfa+23i2AipqAnbbHerLThCHxkxJJQvS7 k64A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710493196; x=1711097996; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=8OmIAT24hWmP9ZRPzXqyUUK9u26w0CUFTu8E1G/NvBg=; b=l3zauheJW+o08jVpJo3ZzuJUUgi6qdh/41T36U2WZAwoWTXc7xLLPMnVo0qB0T2DNy vnmvXEUGm58chGnySSThJXoXnUMWQ0LWHd48rwh9FYEOrFvYz0bEwwoQKKeen0vVeNg2 ljse+2XVZNvC/wJtlFspYjz+YO/7PVMFVQPNz5PflYKgbDZyir2uuhark9RcxfGVG6eP ZP2sP/Xag1gOVTVaWmKmZt6GYNafSGgmLQ0nHKs4aV5M+3m1qzuQ9PIn9KZ9Kyp44Yue 6VDvunhg0MbD2p3ao2cziRbF9eZpQn1Wx9KRvMXcTsYHvLQgCkNXo7H/4BbhnQTKs3EI 4n7Q== X-Forwarded-Encrypted: i=1; AJvYcCX7eSfIQ5tjlnO/4RqSuNi4oOGvwKKUPzyJThRdOyM5PRGkEbzYJD4UM25/Kguzh98nOXmw+vcUE0Lts80DuzmRVOdmEe9ybAjYElHhZOBXmzl+BVn4lIKuR0UBMg== X-Gm-Message-State: AOJu0YwvGWi58Z6P4GoilTxd2jhCxDpt+sMW73ZRZYNAtEnQ2gdMG909 8LYkeS0JxtOi/4458C6+if5k1xk10VLQf2gFjEyHlGVqJtY7aYK1Iwf8nkeI+pvGwKDo4DzBmAb GiFktH3D3vLolCIjaZsWNAx4D0GM= X-Google-Smtp-Source: AGHT+IHNAJk9qF5CLJLyFimbVjm+bLNHreEPpadXunN70GZBtVaF90RGmlI3iM2xvI5bHK9p7Dzu+oGDVkn22SFTido= X-Received: by 2002:a05:6a21:e85:b0:1a1:4d7c:bdc4 with SMTP id ma5-20020a056a210e8500b001a14d7cbdc4mr4014341pzb.22.1710493196436; Fri, 15 Mar 2024 01:59:56 -0700 (PDT) List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org MIME-Version: 1.0 References: <9D19D8E3-5B72-4006-9296-C7D74E670A12@yahoo.com> <867B8EBD-820C-41B6-9B5D-307AD9506343@yahoo.com> In-Reply-To: <867B8EBD-820C-41B6-9B5D-307AD9506343@yahoo.com> From: Oliver Epper Date: Fri, 15 Mar 2024 09:59:45 +0100 Message-ID: Subject: Re: 4-core arm armv7-package-building configuration notes, on RPi4B (aarch64) and OrangePi+2ed (armv7), poudriere-devel based To: Mark Millard Cc: FreeBSD Mailing List , FreeBSD ARM List , Nuno Teixeira Content-Type: multipart/alternative; boundary="0000000000005d4c600613af3b3c" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4Twyqy065zz4r85 --0000000000005d4c600613af3b3c Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I can only suggest deploying a Mac Mini M1 with qemu using Apple HVF as a poudriere server for aarch64. See https://oliver-epper.de/posts/poudriere-on-m1-mac/ I never had success with qemu emulation on amd64 and the Mac Mini builds rust and llvm without problems. I don't think I'd like to wait for these packages to build on a pi. greetings Oliver Am Fr., 15. M=C3=A4rz 2024 um 09:39 Uhr schrieb Mark Millard : > On Mar 12, 2024, at 23:57, Mark Millard wrote: > > > This note's structure: > > > > 1st: Package-build time frame summaries. > > (But I note some hardware points that are repeated later as well.) > > > > 2nd: Configuration points common to both RPi4B and OrangePi+2ed context= s. > > > > 3rd: Configuration points unique to the RPi4B context. > > > > 4th: Configuration points unique to the OrangePi+2ed context. > > > > > > 1st: Package-build time Summaries follow. > > (Note: the detail order of package builds is not the same.) > > (Examples are visiable in these summaries.) > > > > > > RPi4B: cortex-a72 (aarch64) with cortex-a7 (armv7) support, 2 GHz > (overclocked), 8 GiBytes RAM, USB3 > > [00:25:32] [01] [00:13:33] Finished lang/perl5.36 | perl5-5.36.3_1: > Success > > [01:58:13] [02] [00:44:25] Finished devel/icu | icu-74.2,1: Success > > [03:14:00] [02] [00:21:28] Finished lang/ruby31 | ruby-3.1.4_1,1: Succe= ss > > [03:33:51] [01] [02:21:22] Finished devel/cmake-core | > cmake-core-3.28.3: Success > > [23:12:47] [02] [19:06:01] Finished lang/rust | rust-1.76.0: Success > > [1D:00:14:46] [02] [00:55:46] Finished devel/binutils@native | > binutils-2.40_5,1: Success > > (Note: start of visible ordering differences:) > > [1D:03:07:32] [02] [00:58:03] Finished devel/arm-none-eabi-gcc | > arm-none-eabi-gcc-11.3.0_3: Success > > [1D:03:42:09] [01] [1D:00:08:13] Finished devel/llvm18@default | > llvm18-18.1.0.r3: Success > > [1D:04:45:14] [02] [01:35:29] Finished lang/gcc13 | gcc13-13.2.0_4: > Success > > [1D:05:21:43] [01] [01:39:13] Finished devel/boost-libs | > boost-libs-1.84.0: Success > > [1D:05:43:24] [01] [00:21:33] Finished textproc/source-highlight | > source-highlight-3.1.9_9: Success > > [1D:05:47:01] [02] [00:44:22] Finished devel/aarch64-none-elf-gcc | > aarch64-none-elf-gcc-11.3.0_3: Success > > [1D:07:23:25] [02] [01:21:04] Finished devel/gdb@py39 | gdb-14.1_2: > Success > > [1D:07:58:37] [01] [01:19:55] Finished devel/freebsd-gcc13@armv7 | > armv7-gcc13-13.2.0_1: Success > > [1D:07:58:43] Stopping 2 builders > > [main-CA7-default] [2024-03-11_15h30m14s] [committing] Queued: 265 > Built: 265 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 Tobuild: 0 > Time: 1D:07:58:46 > > > > Note: 4364Mi MaxObs(Act+Wir+Lndry+SwapUsed) ("MaxObs": short for > "Maximum Observed") > > Note: SwapUsed maximum: 0 (none used). > > > > So, for an 8 GiByte RAM RPI4B, RAM+SWAP configured to be 38 GiBytes or > so: > > Estmate: 38.0 GiBytes/4.3 GiBytes approx.=3D=3D 8.8 > > Result: Lots of margin for builds that use more RAM+SWAP. > > > > So, for an 4 GiByte RAM RPI4B, RAM+SWAP configured to be 18 GiBytes or > so: > > Estimate: 18.0 GiBytes/4.3 GiBytes approx.=3D=3D 4.1 > > Result: Also lots of margin for builds that use more RAM+SWAP. > > I did the experiment of trying PARALLEL_JOBS=3D3 instead of > PARALLEL_JOBS=3D2 , still MAKE_JOBS_NUMBER_LIMIT=3D3 . It took > a little longer: > > [1D:08:52:32] Stopping 3 builders > [main-CA7-default] [2024-03-13_16h27m18s] [committing] Queued: 265 Built: > 265 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 Tobuild: 0 Time= : > 1D:08:52:35 > > (As the load averages are significantly more than the > available hardware thread count and vary significantly, > comparing individual package build times is not > particularly useful. So I'm not reporting any example > times for packages.) > > At some point I'll likely try the PARALLEL_JOBS=3D2 > MAKE_JOBS_NUMBER_LIMIT=3D3 combination on a RPI4B that has > 4 GiBytes of RAM. I really need the memory pressure > involved in significant paging to get reasonable estimates > for RAM+SWAP requiments, avoiding just ending up with a > large Inact accumulation with an unknown mix of dirty pages > and clean pages. > > But, I'll likely try the RPi5 (8 GiBytes) first, now > that the RPi5 EDK2 has fixed what the problem was that lead > to unreliable USB I/O for UEFI/ACPI. (I'll likely use an > artifact build since the release build looks to not be > present yet.) > > > OrangePi+2ed: cortex-a7 armv7, 1GHz, 4 cores, 2 GiBytes RAM, USB2: > > [01:51:31] [01] [01:00:07] Finished lang/perl5.36 | perl5-5.36.3_1: > Success > > [08:55:35] [02] [03:08:09] Finished devel/icu | icu-74.2,1: Success > > [13:17:38] [02] [01:28:32] Finished lang/ruby31 | ruby-3.1.4_1,1: Succe= ss > > [14:17:44] [01] [09:20:55] Finished devel/cmake-core | > cmake-core-3.28.3: Success > > [4D:01:03:43] [02] [3D:08:48:53] Finished lang/rust | rust-1.76.0: > Success > > [4D:06:26:24] [02] [03:09:35] Finished devel/binutils@native | > binutils-2.40_5,1: Success > > (Note: start of visible ordering differences:) > > [4D:14:54:31] [02] [03:38:55] Finished devel/aarch64-none-elf-gcc | > aarch64-none-elf-gcc-11.3.0_3: Success > > [4D:16:13:00] [01] [4D:01:55:03] Finished devel/llvm18@default | > llvm18-18.1.0.r3: Success > > [4D:18:05:58] [02] [03:11:00] Finished devel/arm-none-eabi-gcc | > arm-none-eabi-gcc-11.3.0_3: Success > > [4D:23:00:13] [01] [06:46:06] Finished devel/boost-libs | > boost-libs-1.84.0: Success > > [5D:00:16:39] [01] [01:15:53] Finished textproc/source-highlight | > source-highlight-3.1.9_9: Success > > [5D:01:17:24] [02] [07:10:52] Finished lang/gcc13 | gcc13-13.2.0_4: > Success > > [5D:09:38:14] [01] [05:56:48] Finished devel/freebsd-gcc13@armv7 | > armv7-gcc13-13.2.0_1: Success > > [5D:10:18:58] [02] [05:44:02] Finished devel/gdb@py39 | gdb-14.1_2: > Success > > [5D:10:31:56] Stopping 2 builders > > [main-CA7-default] [2024-03-06_03h15m10s] [committing] Queued: 265 > Built: 265 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 Tobuild: 0 > Time: 5D:10:31:55 > > > > (So, a little over 4 days longer than the RPi4B example above.) > > > > Note: 2794Mi MaxObs(Act+Wir+Lndry+SwapUsed) ("MaxObs": short for > "Maximum Observed") > > > > > > 2nd: Configuration points common to both the RPi4B and the > > OrangePi+2ed contexts. > > > > ports-mgmt/poudriere-devel is used to build the packages. > > > > devel/llvm18 options: using BE_NATIVE and omitting MLIR. > > (What I normally build for armv7 and aarch64 targetting.) > > > > Also, ports-mgmt/poudriere-devel omits the QEMU option, > > as is normal for me. > > > > 265 packages are built, including pkg. It is the same > > 265 pacakges across contexts. (The order of the builds > > does vary.) > > > > /usr/local/etc/poudriere.conf has . . . > > > > NO_ZFS=3Dyes > > PARALLEL_JOBS=3D2 > > ALLOW_MAKE_JOBS=3Dyes > > MAX_EXECUTION_TIME=3D432000 > > NOHANG_TIME=3D432000 > > MAX_EXECUTION_TIME_EXTRACT=3D14400 > > MAX_EXECUTION_TIME_INSTALL=3D14400 > > MAX_EXECUTION_TIME_PACKAGE=3D57600 > > MAX_EXECUTION_TIME_DEINSTALL=3D14400 > > > > NOTE: MAKE_JOBS_NUMBER_LIMIT is used to constrain > > what ALLOW_MAKE_JOBS does but is not set the > > same across the contexts. > > > > /etc/fstab does not specify any tmpfs use or the > > like: avoids competing for RAM+SWAP. > > > > poudriere armv7 jail worlds are duplicates of each > > other across the different media. Those worlds are > > from a personal buildworld based on using > > -mcpu=3Dcortex-a7 for the code generation. The package > > builds also use that. > > > > /boot/loader.conf has . . . > > > > # Delay when persistent low free RAM leads to > > # Out Of Memory killing of processes: > > vm.pageout_oom_seq=3D120 > > > > Heatsinks and fans for keeping things cool over the > > sustained build activity. > > > > > > 3rd: Configuration points unique to the RPi4B context. > > > > /usr/local/etc/poudriere.conf has . . . > > > > USE_TMPFS=3D"data" > > > > (Based on the larger RAM and RAM+SWAP and that it > > does not grow to be huge for the likes of lang/rust .) > > > > /usr/local/etc/poudriere.d/make.conf has . . . > > > > MAKE_JOBS_NUMBER_LIMIT=3D3 > > > > (Based on the larger RAM and RAM+SWAP.) This does mean > > that the 3 load averages can be 6+ at times on the 4 > > hardware thread system while both ports being built are > > respecting the limit. Some ports do not fully respect > > the limit the whole time. This can make build-times > > a somewhat messier comparison than one might hope across > > the contexts. But for the specifics here, things should > > be clear enough. > > > > RAM =3D=3D 8 GiBytes > > RAM+SWAP =3D=3D 38 GiBytes > > (Note aarch64 allows a larger RAM multiplier limit without > > warning of potential swap-related mistuning: "total > > configured swap (? pages) exceeds maximum recommended > > amount (? pages)" with "increase kern.maxswzone or reduce > > amount of swap".) > > > > 5.1V 3.5A power supply, so a little extra margin for current. > > > > /boot/efi/config.txt has: > > > > over_voltage=3D6 > > arm_freq=3D2000 > > sdram_freq_min=3D3200 > > force_turbo=3D1 > > (Reliable operation, with margin, on the mix of v1.1, v1.4, and v1.5 > > RPi4B's that I have access to, 8 total.) > > > > So: 2 GHz overclocking, using a fixed rate. > > > > USB3 media: U2 Optane 960 GB media via a powered USB3 adaptor. > > > > Kernel has: "arm64: improve UVA layout for 32bit processes" > > ( main's 967022aa5aa6 ). So an armv7 process can be somewhat > > over 3 GiBytes for its address space. > > > > Boot aarch64 env: a PkgBase world and kernel.GENERIC-NODEBUG pair. > > FYI: > > > > # uname -apKU > > FreeBSD aarch64-main-pkgs 15.0-CURRENT FreeBSD 15.0-CURRENT > main-n268514-61b88a230bac GENERIC-NODEBUG arm64 aarch64 1500014 1500014 > > > > > > 4th: Configuration points unique to the OrangePi+2ed context. > > > > /usr/local/etc/poudriere.conf has . . . > > > > USE_TMPFS=3Dno > > > > (Based on the smaller RAM --and smaller RAM+SWAP for avoiding > > potential-mistuning notices.) > > > > /usr/local/etc/poudriere.d/make.conf has . . . > > > > MAKE_JOBS_NUMBER_LIMIT=3D2 > > > > (Based on the smaller RAM --and smaller RAM+SWAP for avoiding > > potential-mistuning notices-- but wanting to still have margin > > for bigger peak RAM+SWAP use than the example happens to do.) > > > > RAM =3D=3D 2 GiBytes > > RAM+SWAP =3D=3D 5.6 GiBytes > > (Note armv7 has a smaller RAM multiplier limit without > > warning of potential swap-related mistuning: "total > > configured swap (? pages) exceeds maximum recommended > > amount (? pages)" with "increase kern.maxswzone or reduce > > amount of swap".) > > > > In /etc/rc.conf I have: > > > > if [ "`sysctl -i -n hw.fdt.model`" =3D=3D "Xunlong Orange Pi Plus 2E" ]= ; then > > sysctl dev.cpu.0.freq=3D1008 > /dev/null > > fi > > > > In other words: a fixed 1GHz or so clock rate is used. > > > > USB2 media: Actually USB3 media that also supports USB2 > > use. 1 TB Samsung Touch T7 (NVMe based) via a powered hub, > > a USB3-capable one. > > > > > > > > Side note: > > > > I've no clue how to judge any tradeoff consequences for > > "increase kern.maxswzone" for judging reasonableness of > > such an action. > > > > > > =3D=3D=3D > Mark Millard > marklmi at yahoo.com > > > --0000000000005d4c600613af3b3c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I can only suggest deploying a Mac Mini M1 with qemu using= Apple HVF as a poudriere server for aarch64.

I never had success wit= h qemu emulation on amd64 and the Mac Mini builds rust and llvm without pro= blems. I don't think I'd like to wait for these packages to build o= n a pi.

greetings
Oliver

=
Am Fr., 15= . M=C3=A4rz 2024 um 09:39=C2=A0Uhr schrieb Mark Millard <marklmi@yahoo.com>:
= On Mar 12, 2024, at 23:57, Mark Millard <marklmi@yahoo.com> wrote:

> This note's structure:
>
> 1st: Package-build time frame summaries.
>=C2=A0 =C2=A0 =C2=A0(But I note some hardware points that are repeated = later as well.)
>
> 2nd: Configuration points common to both RPi4B and OrangePi+2ed contex= ts.
>
> 3rd: Configuration points unique to the RPi4B context.
>
> 4th: Configuration points unique to the OrangePi+2ed context.
>
>
> 1st: Package-build time Summaries follow.
> (Note: the detail order of package builds is not the same.)
> (Examples are visiable in these summaries.)
>
>
> RPi4B: cortex-a72 (aarch64) with cortex-a7 (armv7) support, 2 GHz (ove= rclocked), 8 GiBytes RAM, USB3
> [00:25:32] [01] [00:13:33] Finished lang/perl5.36 | perl5-5.36.3_1: Su= ccess
> [01:58:13] [02] [00:44:25] Finished devel/icu | icu-74.2,1: Success > [03:14:00] [02] [00:21:28] Finished lang/ruby31 | ruby-3.1.4_1,1: Succ= ess
> [03:33:51] [01] [02:21:22] Finished devel/cmake-core | cmake-core-3.28= .3: Success
> [23:12:47] [02] [19:06:01] Finished lang/rust | rust-1.76.0: Success > [1D:00:14:46] [02] [00:55:46] Finished devel/binutils@native | binutil= s-2.40_5,1: Success
> (Note: start of visible ordering differences:)
> [1D:03:07:32] [02] [00:58:03] Finished devel/arm-none-eabi-gcc | arm-n= one-eabi-gcc-11.3.0_3: Success
> [1D:03:42:09] [01] [1D:00:08:13] Finished devel/llvm18@default | llvm1= 8-18.1.0.r3: Success
> [1D:04:45:14] [02] [01:35:29] Finished lang/gcc13 | gcc13-13.2.0_4: Su= ccess
> [1D:05:21:43] [01] [01:39:13] Finished devel/boost-libs | boost-libs-1= .84.0: Success
> [1D:05:43:24] [01] [00:21:33] Finished textproc/source-highlight | sou= rce-highlight-3.1.9_9: Success
> [1D:05:47:01] [02] [00:44:22] Finished devel/aarch64-none-elf-gcc | aa= rch64-none-elf-gcc-11.3.0_3: Success
> [1D:07:23:25] [02] [01:21:04] Finished devel/gdb@py39 | gdb-14.1_2: Su= ccess
> [1D:07:58:37] [01] [01:19:55] Finished devel/freebsd-gcc13@armv7 | arm= v7-gcc13-13.2.0_1: Success
> [1D:07:58:43] Stopping 2 builders
> [main-CA7-default] [2024-03-11_15h30m14s] [committing] Queued: 265 Bui= lt: 265 Failed: 0=C2=A0 =C2=A0Skipped: 0=C2=A0 =C2=A0Ignored: 0=C2=A0 =C2= =A0Fetched: 0=C2=A0 =C2=A0Tobuild: 0=C2=A0 =C2=A0 Time: 1D:07:58:46
>
> Note: 4364Mi MaxObs(Act+Wir+Lndry+SwapUsed) ("MaxObs":=C2=A0= short for "Maximum Observed")
> Note: SwapUsed maximum: 0 (none used).
>
> So, for an 8 GiByte RAM RPI4B, RAM+SWAP configured to be 38 GiBytes or= so:
> Estmate: 38.0 GiBytes/4.3 GiBytes approx.=3D=3D 8.8
> Result: Lots of margin for builds that use more RAM+SWAP.
>
> So, for an 4 GiByte RAM RPI4B, RAM+SWAP configured to be 18 GiBytes or= so:
> Estimate: 18.0 GiBytes/4.3 GiBytes approx.=3D=3D 4.1
> Result: Also lots of margin for builds that use more RAM+SWAP.

I did the experiment of trying PARALLEL_JOBS=3D3 instead of
PARALLEL_JOBS=3D2 , still MAKE_JOBS_NUMBER_LIMIT=3D3 . It took
a little longer:

[1D:08:52:32] Stopping 3 builders
[main-CA7-default] [2024-03-13_16h27m18s] [committing] Queued: 265 Built: 2= 65 Failed: 0=C2=A0 =C2=A0Skipped: 0=C2=A0 =C2=A0Ignored: 0=C2=A0 =C2=A0Fetc= hed: 0=C2=A0 =C2=A0Tobuild: 0=C2=A0 =C2=A0 Time: 1D:08:52:35

(As the load averages are significantly more than the
available hardware thread count and vary significantly,
comparing individual package build times is not
particularly useful. So I'm not reporting any example
times for packages.)

At some point I'll likely try the PARALLEL_JOBS=3D2
MAKE_JOBS_NUMBER_LIMIT=3D3 combination on a RPI4B that has
4 GiBytes of RAM. I really=C2=A0 need the memory pressure
involved in significant paging to get reasonable estimates
for RAM+SWAP requiments, avoiding just ending up with a
large Inact accumulation with an unknown mix of dirty pages
and clean pages.

But, I'll likely try the RPi5 (8 GiBytes) first, now
that the RPi5 EDK2 has fixed what the problem was that lead
to unreliable USB I/O for UEFI/ACPI. (I'll likely use an
artifact build since the release build looks to not be
present yet.)

> OrangePi+2ed: cortex-a7 armv7, 1GHz, 4 cores, 2 GiBytes RAM, USB2:
> [01:51:31] [01] [01:00:07] Finished lang/perl5.36 | perl5-5.36.3_1: Su= ccess
> [08:55:35] [02] [03:08:09] Finished devel/icu | icu-74.2,1: Success > [13:17:38] [02] [01:28:32] Finished lang/ruby31 | ruby-3.1.4_1,1: Succ= ess
> [14:17:44] [01] [09:20:55] Finished devel/cmake-core | cmake-core-3.28= .3: Success
> [4D:01:03:43] [02] [3D:08:48:53] Finished lang/rust | rust-1.76.0: Suc= cess
> [4D:06:26:24] [02] [03:09:35] Finished devel/binutils@native | binutil= s-2.40_5,1: Success
> (Note: start of visible ordering differences:)
> [4D:14:54:31] [02] [03:38:55] Finished devel/aarch64-none-elf-gcc | aa= rch64-none-elf-gcc-11.3.0_3: Success
> [4D:16:13:00] [01] [4D:01:55:03] Finished devel/llvm18@default | llvm1= 8-18.1.0.r3: Success
> [4D:18:05:58] [02] [03:11:00] Finished devel/arm-none-eabi-gcc | arm-n= one-eabi-gcc-11.3.0_3: Success
> [4D:23:00:13] [01] [06:46:06] Finished devel/boost-libs | boost-libs-1= .84.0: Success
> [5D:00:16:39] [01] [01:15:53] Finished textproc/source-highlight | sou= rce-highlight-3.1.9_9: Success
> [5D:01:17:24] [02] [07:10:52] Finished lang/gcc13 | gcc13-13.2.0_4: Su= ccess
> [5D:09:38:14] [01] [05:56:48] Finished devel/freebsd-gcc13@armv7 | arm= v7-gcc13-13.2.0_1: Success
> [5D:10:18:58] [02] [05:44:02] Finished devel/gdb@py39 | gdb-14.1_2: Su= ccess
> [5D:10:31:56] Stopping 2 builders
> [main-CA7-default] [2024-03-06_03h15m10s] [committing] Queued: 265 Bui= lt: 265 Failed: 0=C2=A0 =C2=A0Skipped: 0=C2=A0 =C2=A0Ignored: 0=C2=A0 =C2= =A0Fetched: 0=C2=A0 =C2=A0Tobuild: 0=C2=A0 =C2=A0 Time: 5D:10:31:55
>
> (So, a little over 4 days longer than the RPi4B example above.)
>
> Note: 2794Mi MaxObs(Act+Wir+Lndry+SwapUsed) ("MaxObs":=C2=A0= short for "Maximum Observed")
>
>
> 2nd: Configuration points common to both the RPi4B and the
>=C2=A0 =C2=A0 =C2=A0OrangePi+2ed contexts.
>
> ports-mgmt/poudriere-devel is used to build the packages.
>
> devel/llvm18 options: using BE_NATIVE and omitting MLIR.
> (What I normally build for armv7 and aarch64 targetting.)
>
> Also, ports-mgmt/poudriere-devel omits the QEMU option,
> as is normal for me.
>
> 265 packages are built, including pkg. It is the same
> 265 pacakges across contexts. (The order of the builds
> does vary.)
>
> /usr/local/etc/poudriere.conf has . . .
>
> NO_ZFS=3Dyes
> PARALLEL_JOBS=3D2
> ALLOW_MAKE_JOBS=3Dyes
> MAX_EXECUTION_TIME=3D432000
> NOHANG_TIME=3D432000
> MAX_EXECUTION_TIME_EXTRACT=3D14400
> MAX_EXECUTION_TIME_INSTALL=3D14400
> MAX_EXECUTION_TIME_PACKAGE=3D57600
> MAX_EXECUTION_TIME_DEINSTALL=3D14400
>
> NOTE: MAKE_JOBS_NUMBER_LIMIT is used to constrain
>=C2=A0 =C2=A0 =C2=A0 what ALLOW_MAKE_JOBS does but is not set the
>=C2=A0 =C2=A0 =C2=A0 same across the contexts.
>
> /etc/fstab does not specify any tmpfs use or the
> like: avoids competing for RAM+SWAP.
>
> poudriere armv7 jail worlds are duplicates of each
> other across the different media. Those worlds are
> from a personal buildworld based on using
> -mcpu=3Dcortex-a7 for the code generation. The package
> builds also use that.
>
> /boot/loader.conf has . . .
>
> # Delay when persistent low free RAM leads to
> # Out Of Memory killing of processes:
> vm.pageout_oom_seq=3D120
>
> Heatsinks and fans for keeping things cool over the
> sustained build activity.
>
>
> 3rd: Configuration points unique to the RPi4B context.
>
> /usr/local/etc/poudriere.conf has . . .
>
> USE_TMPFS=3D"data"
>
> (Based on the larger RAM and RAM+SWAP and that it
> does not grow to be huge for the likes of lang/rust .)
>
> /usr/local/etc/poudriere.d/make.conf has . . .
>
> MAKE_JOBS_NUMBER_LIMIT=3D3
>
> (Based on the larger RAM and RAM+SWAP.) This does mean
> that the 3 load averages can be 6+ at times on the 4
> hardware thread system while both ports being built are
> respecting the limit. Some ports do not fully respect
> the limit the whole time. This can make build-times
> a somewhat messier comparison than one might hope across
> the contexts. But for the specifics here, things should
> be clear enough.
>
> RAM =3D=3D 8 GiBytes
> RAM+SWAP =3D=3D 38 GiBytes
> (Note aarch64 allows a larger RAM multiplier limit without
> warning of potential swap-related mistuning: "total
> configured swap (? pages) exceeds maximum recommended
> amount (? pages)" with "increase kern.maxswzone or reduce > amount of swap".)
>
> 5.1V 3.5A power supply, so a little extra margin for current.
>
> /boot/efi/config.txt has:
>
> over_voltage=3D6
> arm_freq=3D2000
> sdram_freq_min=3D3200
> force_turbo=3D1
> (Reliable operation, with margin, on the mix of v1.1, v1.4, and v1.5 > RPi4B's that I have access to, 8 total.)
>
> So: 2 GHz overclocking, using a fixed rate.
>
> USB3 media: U2 Optane 960 GB media via a powered USB3 adaptor.
>
> Kernel has: "arm64: improve UVA layout for 32bit processes"<= br> > ( main's 967022aa5aa6 ). So an armv7 process can be somewhat
> over 3 GiBytes for its address space.
>
> Boot aarch64 env: a PkgBase world and kernel.GENERIC-NODEBUG pair.
> FYI:
>
> # uname -apKU
> FreeBSD aarch64-main-pkgs 15.0-CURRENT FreeBSD 15.0-CURRENT main-n2685= 14-61b88a230bac GENERIC-NODEBUG arm64 aarch64 1500014 1500014
>
>
> 4th: Configuration points unique to the OrangePi+2ed context.
>
> /usr/local/etc/poudriere.conf has . . .
>
> USE_TMPFS=3Dno
>
> (Based on the smaller RAM --and smaller RAM+SWAP for avoiding
> potential-mistuning notices.)
>
> /usr/local/etc/poudriere.d/make.conf has . . .
>
> MAKE_JOBS_NUMBER_LIMIT=3D2
>
> (Based on the smaller RAM --and smaller RAM+SWAP for avoiding
> potential-mistuning notices-- but wanting to still have margin
> for bigger peak RAM+SWAP use than the example happens to do.)
>
> RAM =3D=3D 2 GiBytes
> RAM+SWAP =3D=3D 5.6 GiBytes
> (Note armv7 has a smaller RAM multiplier limit without
> warning of potential swap-related mistuning: "total
> configured swap (? pages) exceeds maximum recommended
> amount (? pages)" with "increase kern.maxswzone or reduce > amount of swap".)
>
> In /etc/rc.conf I have:
>
> if [ "`sysctl -i -n hw.fdt.model`" =3D=3D "Xunlong Oran= ge Pi Plus 2E" ]; then
> sysctl dev.cpu.0.freq=3D1008 > /dev/null
> fi
>
> In other words: a fixed 1GHz or so clock rate is used.
>
> USB2 media: Actually USB3 media that also supports USB2
> use. 1 TB Samsung Touch T7 (NVMe based) via a powered hub,
> a USB3-capable one.
>
>
>
> Side note:
>
> I've no clue how to judge any tradeoff consequences for
> "increase kern.maxswzone" for judging reasonableness of
> such an action.
>



=3D=3D=3D
Mark Millard
marklmi at yahoo.com


--0000000000005d4c600613af3b3c--