From nobody Thu Feb 29 19:00:30 2024 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Tm0t654j2z5Cjwb for ; Thu, 29 Feb 2024 19:00:46 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-54.consmr.mail.gq1.yahoo.com (sonic316-54.consmr.mail.gq1.yahoo.com [98.137.69.30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Tm0t54dBxz4TWj for ; Thu, 29 Feb 2024 19:00:45 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=myckckhT; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.30 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1709233242; bh=EkAIQ581ch2r5i8oQfiyHZv/t+WAyIP9nreeivF+syQ=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=myckckhTfgJwZ1MZ1C7TZ8Er8bR05HvaAUeoU59d8P/AXwRyfhDq6XbyOlP2jWj/AtIYZWI/LDbRN7UM49jFe5mMMrGKzetS0/JJ/x+2jjJXjoYMs70NFqQmstdSKtqMKtHx8kMi06vR9i7U8yDRjqfap7s40xxxkWSaWGXNBJvblomYmpXzmh1HqxavlcLKSd4cg+jCe5Qy2WTkmyI+B7E0ucsalQ6G2+DyYn+O+rJYv5l8C8l75CRx7X4td17TDsnMIQahSemcZgPbH6A5fg9Nfw5c19VgMiIEQ72vf6S+30jI39oFr23aAKixw/fI04+JKfRxNN5kBEZgCqaEJg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1709233242; bh=MhqfrM7b23nSUMcZmwG/Xx/6qqAhm9MFIsseNIlLuqu=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=VCBFJQ0e/LDULvT5Y2IpOwPnXT48anBqSNqktQmZGZ3pCsEn6zOUw1PcrJ1zj5tTrqG8S+CTJSlm2DDAims6DOadG/YQYfZewP1Jst9BQjBCqaiwweHfjNBvoz+vt9EW8djn40ULySYsX9p+mpIJ0vY3+kewoYOWNBTaBg0uRIsQ1cbbqF/+LQ7zgoVa1EQKlhy9E8aMIvTI5b8YL5y5M5VFkKQylv2QSISqq61AVWEEplWXIMx/sOuaey6Y5vUegcL3m2ZU6GKSNoaA2hcW/DwG/QuapWvrf5Y8YeU2964YnZeam3y1/zcwYAKBIvQyBmLX3G38b7Th7LWswQgpDw== X-YMail-OSG: voijj.kVM1m03kEda.tNizZOafev3_buKlTbwsEMUaHfYdTzbxSX_2ZWB_vd6BS m5_TI0oqvsCnXP2XgOCB1XdA252YKe9CL4eOUGMN51ZUY0ughDeRhW2OnYxgyPGmcnRzrHG1FZVV _m8qlm8TpAcL7N88Ztah0o56c7nDNQEFOXbV4Nr79tC6SWk_l90fqwTlhs4NB9xAKL70boo6FAFC SLhwFODmkGX6kzp849T0fN52NYXjuffz_2LNIdHPu.MTEHCyIVZgT7ObJzMFn0Y7tMLn.kmzTRB5 4wYt0v1vhVEeEdSBfZ7o0shbzHQTE94sFiUZ.HnHwtGCMreG.dnEmHyzbS9y.2_LUHxk8IFzIts7 tB9pSevR7lANMViYdvzPYUyyV_HaCVtvHCjF9LY.NfWi87GjP9SqzaCRAP4J0duPFDZ_fNYAZwBF p3CnP3G1K_gCb_ieONZayk2B.ZppL1OT60iMAktM29nazmu46R2K6wktQj_Q4o98ls4Qu4MOEGJD 0zsU1ce9t4ZSKcfMHRm2ESfldthnYzJUYnE5wEu1htUALGkK.ctR7RuB3foM7q59yEapfJyW2An2 cUsDHzDdZ6bgscdsYjyBuHeMq2mJn0f44VCo76hj2UIQrDV2I9Q5kxsq3VKfRrtInaDB4YfjXEwB Z2LuEbsQr4e_7MOBClm7f9rAlva.q0SrLtBMfgiWdhL.rzv5cM3pI0ZfS6SKe4GjEP6qJaaCkOsQ V.rVFAQvfifEjWrpo5E0DqsBH.18K62WnXBP58shP12FUSuL6X2VmmgvPQo4BdhClUz2.a9ay_mU sdth8Vjkzh5mNgjQKrrfkvc4.zQXAqzp69v88E9sWSW5f5GT73DLDU8sBvaMzbZhL4NY9yo0jff. cRVzh8WdLWT.2lq180C1Dziv29ibuAKNbaamQ0gbsl5FhNuR02NuaQIrJzRvtsw5LW5h9MqjR6ZG 6O1tZPWK50GQ0jDWUmauA7NkoE.geZTMhws41q9OaYGJrqm95MWdKdMBINDeQkmNYuYVKS_ENnah uBkdsh_ve4bR5peOt2GU9TL0tykh_ziJqDSWYFuV0fJuAOqrsR_hKvq3XpfIkgwDLxhPPkhFqREe o_E7bVsJ0xljddHGHwsQ23RMPaCYOk.8H.JnHRqRWRcjXWjvOGhDoIP7EP1qrM7CKirSPDg3_6fK LBjwIbj.JcKEodQCiP36zllidWP9nQw6eZXM8FB3n6yrWV.UBpLsTO18bCNJGPChVxNaCnTEWVtD rgSpTGBh7YIFdZ6KuPtUdI_8N42_8SxeDveDejJQhd74L2bJyQIgjascmXzLGftw14T8Tm6hFvyy Zini.remmwDbMT.kIPd_PrBo6_1AOhi9v3__JjF561NQxu_fLSl3LEqEuvpzyWI.9h7DArSDD6gU 90nwG1.pJfEOBVGINZ207o1qramE_VBnJub.w49HYR6_yormde_SzTAWzms5awpn3NaUfdGATc03 lUkc6UWngaE4G7YT9TduKQPxexlsyOvTu2Tnj0.MLIsz47zC0x3FgTAQ1CkcqVFybL99j0Ckq8Um wBOfV11xfPTQF_3.zXaYPPzxxnR7LoIDwacN4Va8Fhlo35TQWpcRkjOwQY0t7a6inLJ85inSl471 1_9Oo00rruadmJDERKgDtbrAp6UQv2YWVRZRkzZamt6afVwQVnf1MbL9fO7Xnj2Di8tYhumf1ocR 8UvLy5EeRX7KhnuvOImI6YsGaV3HlQ3iHj_TC2FmOBnzBo2RBYdO7CXqUIVidhng3Bc4lmz4b4l2 7qhc0ku7A7mvlrEyDnPs5awvZOvhuqQUZR2Evz_68J1xYux421WwaStE1_gfgOaOjhW862MHjw_d 5BnRdnXKN7vcLgvU7bTPNJE6cctBwLWP.JALzBR2FBvOwJixHENOrZZFYYl5v3myKSvkdEnhAGie k6Yf7VyQxZb1k7i.JjSlq4T_SO0fLFrStf8Vrc42Zc15YZjiOkUXOP7oBJ1fZyb7Q.myCoUVQgHF GFuP7bVxN3DSbLOoD7cz4EPoPtMvuJIgwq5_gwwzejkIXZ4qrKyl8uCfJ66zWcpeGIW9WwEeinxH p.6Cuv6rQrxGDRKudXoQm6kxV_6wN7OuaWAEi_2DBUUX_uN2FEecwf9RdKaQ.p4wiU1M7mrfKssK HK5gcurJSkBY.HDddJxc0bAakfzgyYficSt7McNGo7yGeLz7WP5oc55ihrj9BAM4HqcbStzhIb4E 8o_zO3antyXYdxWiahl8qkP2IjzmGPCx4WbRD1w_ig52u5Ey.Wmg5If8.R2W_VMJ28gWv0j6S5af FSbg- X-Sonic-MF: X-Sonic-ID: 580e88a1-e83f-476b-a6e7-68a61460d1e3 Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Thu, 29 Feb 2024 19:00:42 +0000 Received: by hermes--production-gq1-5c57879fdf-7xbd4 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 203250b4d96bdbdc74fe0e6cdfa91257; Thu, 29 Feb 2024 19:00:40 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: Re: How to investigate an unexpected port build time-taken relationship in an aarch64 context? Date: Thu, 29 Feb 2024 11:00:30 -0800 References: <7D640D5A-7514-480E-8D5B-58003DB558E1@yahoo.com> <900B51D1-2005-4281-BFBD-96B49B147A13@yahoo.com> To: freebsd-hackers , FreeBSD ARM List In-Reply-To: Message-Id: <5FA20383-1469-439C-9F6E-6707C3447F2B@yahoo.com> X-Mailer: Apple Mail (2.3774.400.31) X-Spamd-Bar: -- X-Spamd-Result: default: False [-3.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.30:from]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.30:from] X-Rspamd-Queue-Id: 4Tm0t54dBxz4TWj [The armv7 bulk build variation of that test proves to be an interesting contrast with a prior result.] On Feb 29, 2024, at 10:04, Mark Millard wrote: > [A different test case also gets the shorter time frame.] >=20 > On Feb 28, 2024, at 22:59, Mark Millard wrote: >=20 >> On Feb 28, 2024, at 18:46, Mark Millard wrote: >>=20 >>> Example HW Context: Windows Development Kit 2023 >>> 8 cores: 4 cortex-A78C's and 4 cortex-X1C's >>> Headless: serial console and ssh access, no x11 or the like = installed. >>> UFS use. >>>=20 >>> Note: cortex-A76's are missing 3 or so instruction set >>> features compared to A78C/X1C parts. Use of >>> -mcpu=3Dcortex-a76 generated code is compatibile (and would allow >>> the code to run on a cortex-a76 system, such as an RPi5 once >>> supported). >>>=20 >>> I've been doing poudriere-devel bulk timing experiments based on: >>>=20 >>> A) PkgBase based system software (kernel and world) and >>> general use of default code generation for ports and >>> such. >>>=20 >>> B) A personal -mcpu=3Dcortex-a76 based kernel, world, port builds >>> (into packages via poudriere-devel). >>>=20 >>> C) Also use of an armv7 poudriere jail based on armv7 PkgBase >>> and default armv7 code generation. This was used in both the >>> (A) and (B) contexts. These also show what I'm curious about. >>>=20 >>> Using the armv7 poudriere jail context for illustration: >>>=20 >>> For (B) used via the armv7 context: >>>=20 >>> [05:40:24] [03] [04:55:38] Finished lang/rust | rust-1.76.0: Success >>> . . . >>> [05:45:58] [01] [05:01:12] Finished devel/llvm18@default | = llvm18-18.1.0.r3: Success >>> [05:46:00] [01] [00:00:00] Building devel/boost-libs | = boost-libs-1.84.0 >>> [06:59:23] [01] [01:13:23] Finished devel/boost-libs | = boost-libs-1.84.0: Success >>>=20 >>> For (A) used via the armv7 poudriere jail context: >>>=20 >>> [06:33:21] [01] [05:40:48] Finished lang/rust | rust-1.76.0: Success >>> . . . >>> [06:40:05] [05] [05:48:09] Finished devel/llvm18@default | = llvm18-18.1.0.r3: Success >>> [06:40:07] [01] [00:00:00] Building devel/boost-libs | = boost-libs-1.84.0 >>> [06:57:48] [01] [00:17:41] Finished devel/boost-libs | = boost-libs-1.84.0: Success >>>=20 >>> The curiosity is about the 01:13:23 vs. 00:17:41 boost-libs: The >>> ratio is large and in the opposite direction to most time trends. >>>=20 >>> Notes: Almost all the time llvm18 and rust were building, both were >>> building but little else did and the load average was 16+ from the >>> llvm18/rust build activity. When boost-libs was building it was the >>> only thing building and it looked to be single threaded when I >>> was watching. >>=20 >> I should have been explicit that the 01:13:23 was mostly >> "stage" phase (not "build" phase) and I was referring to >> the "stage" phase as far as single threaded is concerned. >>=20 >>> (A) and (B) without use of the armv7 context got similar results >>> when I first noticed this but I'm going back and recording times >>> for some variations. I do not have those to report other >>> pairs of results yet. >>>=20 >>> (In the armv7 poudriere jail context reported:) >>> (B) takes less time for llvm18 and rust than (A) does. >>> (A) takes vastly less time for boost-libs than (B) does, >>> approximately a factor of 4 for the time-ratio. >>>=20 >>> I'd be curious to get a clue what contributes to the boost-libs >>> time ratio being so extreme once I have figures for other >>> combinations of poudriere jail content vs. the system's content. >>=20 >> Turns out that for the aarch64 jail (PkgBase system and >> default code generation), stage started about 10 min into >> the boost-libs activity. Package started somewhat under 5 >> minutes later. End result: >>=20 >> [05:55:56] [03] [05:33:12] Finished lang/rust | rust-1.76.0: Success >> . . . >> [06:04:37] [01] [05:41:53] Finished devel/llvm18@default | = llvm18-18.1.0.r3: Success >> [06:04:39] [01] [00:00:00] Building devel/boost-libs | = boost-libs-1.84.0 >> [06:20:50] [01] [00:16:11] Finished devel/boost-libs | = boost-libs-1.84.0: Success >>=20 >> So, very similar to the armv7 jail result for a PkgBase >> system context (aarch64 boot and aarch64/armv7 PkgBase >> jail, default code generation). >>=20 >> It appears that the boost-libs "stage" phase is the context >> for my question. >>=20 >> For the jail code generation being based on -mcpu=3Dcortex-a76 >> code generation but the boot having been PkgBase based: >>=20 >> Stage started about 11.5 min into the boost-libs activity. >> Package started around 48 minutes later. End result >> (showing only boost-libs): >>=20 >> [01:07:01] [01] [01:06:31] Finished devel/boost-libs | = boost-libs-1.84.0: Success >>=20 >> I'll note that bjam stays around 100% CPU in top during this >> much longer "stage" phase. Definitely less than the 01:13:23 >> time. MWCHAN "-", STAT RJ, PRI 135 when I looked. >>=20 >> For the jail code generation and boot context both being based >> on -mcpu=3Dcortex-a76 code generation: >>=20 >> [05:16:38] [01] [00:49:22] Finished devel/boost-libs | = boost-libs-1.83.0_1: Success >>=20 >> Also definitely less than the 01:13:23 time. (I'm showing here an >> earlier test when it was boost-libs v1.83.) >>=20 >> A ZFS context (instead of UFS context) showed: >>=20 >> [04:37:47] [03] [04:03:16] Finished lang/rust | rust-1.76.0: Success >> . . . >> [04:43:47] [01] [04:09:16] Finished devel/llvm18@default | = llvm18-18.1.0.r3: Success >> [04:43:48] [01] [00:00:00] Building devel/boost-libs | = boost-libs-1.84.0 >> [05:41:46] [01] [00:57:58] Finished devel/boost-libs | = boost-libs-1.84.0: Success >=20 > Another test case . . . >=20 > So I booted the -mcpu-cortex-a76 kernel and world (kernel > configures to use LSE atomics) and chroot'd into a aarch64 > PkgBase world that has a PkgBase jail for poudiere use and > that uses default code generation. >=20 > Stage started about 9 min into the boost-libs activity. > Package started around 4.5 minutes later. End result > (showing only boost-libs): >=20 > [00:15:56] [01] [00:15:48] Finished devel/boost-libs | = boost-libs-1.84.0: Success >=20 Given that, I then tried the armv7 poudriere jail from inside the chroot into the PkgBase aarch64, boot kernel and world still being the -mcpu=3Dcortex-a76 and LSE atomics context: [00:15:08] [01] [00:14:57] Finished devel/boost-libs | = boost-libs-1.84.0: Success But my earlier reported test without that chroot involved (so from a -mcpu=3Dcortex-a76 world) was 01:13:23 instead of 00:14:57 : [06:59:23] [01] [01:13:23] Finished devel/boost-libs | = boost-libs-1.84.0: Success So it appears that having the poudriere armv7 jail matched with a closest-containing PkgBase aarch64 world (chroot here), and a kernel using LSE atomics, gets the shorter time frame for the armv7 context. (I've never seen the shorter time frame on a cortex-a72/a53 for which LSE atomics is not an option, non matter if armv7 was involved or not.) =3D=3D=3D Mark Millard marklmi at yahoo.com