From nobody Fri Nov 10 01:26:57 2023 X-Original-To: freebsd-ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SRLln0klRz50Kkm for ; Fri, 10 Nov 2023 01:27:17 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic312-25.consmr.mail.gq1.yahoo.com (sonic312-25.consmr.mail.gq1.yahoo.com [98.137.69.206]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SRLll63Pkz3HRJ for ; Fri, 10 Nov 2023 01:27:15 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=WWkrTREx; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.206 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1699579633; bh=AIr1FeJMkQtLg6CYBBQ7PXdvcfZeaCkE31wWx6fY4J8=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=WWkrTRExgZ3R5CB+lAwbc8MEFdoxkIPPG26zEE1kmSXtLsdpnweVzNt3IHRsDOTiD3slezcDZLad7SjJXbFVIi1qfDvz47zCLiSVe5fIDOQ78kQBNYHu7yoXDwIBiGKQZsXGv3Y5m/171SaWw6RAh7KUzDYxyzFomZTxvSm6XaLVR5+l5CsaiEdtz5fTZF+sCVJJ6wrxl9S0Y5L0qBnrlQlOMift5dLiISOuD4QFtHI8k+/z6AZZqX/a+TxZ9mTMIMjV4lyBDZXTg3aExEBmP2IEWojr2FHzkqJbF6xw2dNOJVmnqPv1VoLzkffG1fIOshzRAsIO8e0M8YlviUSl4w== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1699579633; bh=arFk3g9EUL+RrTx2E4ATFH+LtInyQOVJ1iTFkS3Qsw6=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=Cg5EzWUsyh+kDKLDO4Yee/Pek+BoDzpCKmIbPHVQyVqRU/S3abVGNxbh5BRXbxsWU5pNAKpDdbYUq1QnfU9eg4fO3U3J/jHom+KwStnbugJT0uuUSPNGpS82ugreqhKyKdTSPam6d4NUpdY8La9L/D3B7UvMN5wrGptyHmaFvz0DM/DzdbGVa0osz2gFSUcwBBgweJgozHtpUNgx05Dg27LEJ2yQ58rCadPOotleuVkWIJsMcx5naD8/DupVAkVC8xHRQzVd0F4HPAURhEmXAjzj6w6nxeY4Rdrds6jDlcAzVapRZKIovrM+NdY/Y4GEFRLWsOyyXjD1I+pYj+18Cg== X-YMail-OSG: hyT82dQVM1mktGQLRbOIy9DCFbovFjGbBjUBfJKZJhyrQIl47_MZhYSJ6D7XwmI oSykgBaQjWQHmSqb_pL13O2QN9ltgAXSI.PPAj5NxsGkK0MxzFgF1U7HaACeEHps8artn3OXZAaG zis8VwTTb6rFYMkk7bOHlvj9ssX1xnVi2E1ljlF7OlsNUnD3XiXU8AsNlZjCRjNyMdIw_lRmABuM p81cY7ZvYKGXW7G3.xuJuOa3pCBAF9KK5QNGE5YmNartuGcodidflmoPDkw_QqnPLTIz62rQmCN6 80__xpa2kZ8oVz0WI_Lj3lLpb8PAPwylMXvNO68eyuZSoaUHjD2FNFeI6iCkQg3P4A9oFERgu3ni q6Lbe5dATNpeiOW.KIoduiXTnldQXNYoZEcp0fFeQVKptRWL44D3lSz8aI3mBZSnOCWM.4RzIVN9 aM61gLpf_EqPW2B7rygviaGMxgZF3V0IPgjEmqogwmoTDuYwIar.3FJPJYfooQFwJieDuYR8F8Tk cPaZTqqF9tV8u5E_EtnlWDGEHJbD_WNT6tN7yBbN8TkOoBy64oQ_wFxje7vXSdm.VDpXlpoTqr.E GEzmLG4PtIIhkvx7uH_2msRmUFNcKkezWlLiABqDs2ZtuzgMrRklLyGK78oM5_54AHlk3xS.nyZQ RmdgDfWQeOqPq3N0topUri9mUoZZ2Sf2Kjn4OURVsrkbmlvfqGsWt4fhYAYVjU.Xieg9XkZABveL B4qImZLOl.LFmR4Xw4GxwHDC5aYtvuXx4nEvjUakiIpGhT_CsyAQnqYHq8rSfkvhSg8oE4YwYOuM _I6QeU6crAIc8UblxnYkFUMm8AB3DseA0tXB3iZRFDt16sePTKEulIWDbFf0NqwnazsBEfgq7s0e s7c1nxAG11v2F0U3nI.nLFB5YcDwyo7fm4TwvKPG4pgLub9MDukKVD78TGfqaUuxV3R79GORv1bU T5mLG5LM0noffsc4YeqGsYbCNLkk.M_eKD865fRl2HKerC3mtd0dEpQzdaWqw62.Qd83W8o3XGh6 JbjABSEWGuvnl6awHK2_x3MOv7cxy5VLCslgdH_6St7Mvb6va0cxh7kb8zoluA1UAbvHShkmRsKQ q9ujPHeEG9VdQkavtH2rdkCKmeOQ13kO9KwJOvHneo_70TW.o1mksGhW.dF.9UEVd2aGIUSYQnfM zC2ht.vbum_RWCULGbDdg5F3.xZRKlp6vadaO7SABx94aqLBi5EG4MjHW3fc0cgM1JizCxdlis1K Aj05pwMizr9S83arlMzOuDkUt5g_VEIDuMjqe49922kAev8ZxE.bD3Aes06wOkdqLT0mSpPru97z 5tQy.Jjy81XU7yP2uWIqrtHI0c42t5cPVtg1En1QDUw6fllD9umyiNusOiBse_RVqAiCV1axbGZw zIql.xVo7DzOY6_EdyeHrGwF49b3pSx1D.Lh9OOaGJTmhVOHckDmX7MEAkgNhViGrxolW.tNToCS gxe9yhC6GLaXfgsiE6SHHeV1T22yq4Bv9JuAvQoD2KMLbM8tblBuQU1ASF85EYL6A7vsx4R9XEzm .PQhuLp3VP1m_ghkXxJeDKj6FQetcexRNApHhHA0Y_3STuYsYBz8XhrzQYqu5rUVA.tn3xtER_9x DmlwZ6jfd52oLM_8nQpIUrKgtcGm0dRW1Wq2rfiUmSJukx28DSCsGKcPS6Xki28I6yYN5K1gL2q5 zYK0vlyD.s3.ZG0OR3pAXUBs6yWObA.kRU5SlzzEbm6iLvv6hTqnAfwtvCmJYESryOW6P0lB9LXr To4MkQQfJoQJug8QDvRj4mFVwmYSptcGsn0nuyUVcO0AlhmVonzylVaWnboz5akxQQtd0f5nnDIp VGn3_hSw1qjuHvSZEIJzzrcJDZljKkxhXedjfJks4R6D2uBPZWkBnamF7n.gHSxNElt.Zfg0.F_z llR0RZahUtbj0v5cGh1cpVsTQZONkm.LlqCM6pNb5XJAY6N29FZ6DJARsgwqm6NUNyTNAFNwfX4F fz11rYwZxiAN02reurT6Da0ZgZsiZ5ClCIYZowPf8TNowu_lthrjwTn0bfFiS0VpMIq9anTI.WQL YCjFSiS9XCujk1PB0HaHuNTw1ysQKUVn6cPXw2PbaWCOZLinytBn85XVuublyPEptYBgrzvBZ9Fh aIcFYDmN5f9kajLz4YAOPRtiRmx49TdmGXYNGwsSSUmrazt9.Vs8m09ld2rRHEvMR8w7kI3U0CD2 w7yEt_LyfFk3gF5pAK8UUMKgRCf6W79R.OTRGzipemkgd7KCa6osu4flP4C_ndgmFzXS2uxW8uA- - X-Sonic-MF: X-Sonic-ID: 40ce2be9-5a8c-45c8-81df-cfe2cfc6693f Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Fri, 10 Nov 2023 01:27:13 +0000 Received: by hermes--production-ne1-56df75844-vlcrv (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID d12678b1cee0ee2a8a0c5513cd50b5c9; Fri, 10 Nov 2023 01:27:09 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.200.91.1.1\)) Subject: Ryzen 9 7950X3D bulk -a times: adding an example with SMT disabled (so 16 hardware threads, not 32) Message-Id: <88907269-7ECD-4539-AA3D-AD0A31B13CA7@yahoo.com> Date: Thu, 9 Nov 2023 17:26:57 -0800 To: FreeBSD Hackers , FreeBSD Mailing List X-Mailer: Apple Mail (2.3774.200.91.1.1) References: <88907269-7ECD-4539-AA3D-AD0A31B13CA7.ref@yahoo.com> X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-ports@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.206:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.206:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SRLll63Pkz3HRJ X-Spamd-Bar: --- Reading some benchmark results for compilation activity that showed some SMT vs. not examples and also using my C++ variant of the old HINT benchmark, I ended up curious how a non-SMT from scratch bulk -a would end up (ZFS context) compared my prior SMT based run. I use a high load average style of bulk -a activity that has = USE_TMPFS=3Dall involved. The system has 96 GiBytes of RAM (total across the 2 DIMMs). The original under 1.5 day time definitely had significant swap space = use (RAM+SWAP =3D 96 GiBYtes + 364 GiBytes =3D=3D 460 GiBytes =3D=3D 471040 = MiBytes). The media was (and is) a PCIe based Optane 905P 1.5T. ZFS on a single partition on the single drive, ZFS used just for bectl reasons, not = other typical use-ZFS reasons. I've not controlled the ARC size-range = explicitly. So less swap partition use is part of contribution to the results. The original bulk -a spent a couple of hours at the end where it was just fetching and building textproc/stardict-quick . I have not cleared out /usr/ports/distfiles or updated anything. So fetch time is also a difference here. SMT (32 hardware threads, original bulk -a): [33:10:00] [32] [04:37:23] Finished emulators/libretro-mame | = libretro-mame-20220124_1: Success [35:36:51] [23] [03:44:04] Finished textproc/stardict-quick | = stardict-quick-2.4.2_9: Success . . . [main-amd64-bulk_a-default] [2023-11-01_07h14m50s] [committing:] Queued: = 34683 Built: 33826 Failed: 179 Skipped: 358 Ignored: 320 Fetched: = 0 Tobuild: 0 Time: 35:37:55 Swap-involved MaxObs (Max Observed) figures: 173310Mi MaxObsUsed 256332Mi MaxObs(Act+Lndry+SwapUsed) 265551Mi MaxObs(Act+Wir+Lndry+SwapUsed) (So 265551Mi of 471040Mi RAM+SWAP.) Just-RAM MaxObs figures: 81066Mi MaxObsActive (Given the complications of getting usefully comparable wired figures = for ZFS (ARC): omit.) 94493Mi MaxObs(Act+Wir+Lndry) Note: MaxObs(A+B+C) <=3D MaxObs(A)+MaxObs(B)+MaxObs(C) ALLOW_MAKE_JOBS=3Dyes was used. No explicit restriction on PARALLEL_JOBS or MAKE_JOBS_NUMBER (or analogous). So 32 builders allowed, each allowed 32 make jobs. This explains the high load averages of the bulk -a : load averages . . . MaxObs: 360.70, 267.63, 210.84 (Those need not be all from the same time frame during the bulk -a .) As for the ports vintage: # ~/fbsd-based-on-what-commit.sh -C /usr/ports/ 6ec8e3450b29 (HEAD -> main, freebsd/main, freebsd/HEAD) devel/sdts++: = Mark DEPRECATED Author: Muhammad Moinur Rahman Commit: Muhammad Moinur Rahman CommitDate: 2023-10-21 19:01:38 +0000 branch: main merge-base: 6ec8e3450b29462a590d09fb0b07ed214d456bd5 merge-base: CommitDate: 2023-10-21 19:01:38 +0000 n637598 (--first-parent --count for merge-base) I do have a environment that avoids various LLVM builds taking as long to build : llvm1[3-7] : no MLIR, no FLANG llvm1[4-7] : use BE_NATIVE other llvm* : use defaults (so, no avoidance) I also prevent the builds from using strip on most of the install materials built (not just toolchain materials). non-SMT (16 hardware threads): Note one builder (math/fricas), the last still present, was stuck and I had to kill processes to have it stop unless I was willing to wiat for my large timeout figures. The last builder normal-finish was: [39:48:10] [09] [00:16:23] Finished devel/gcc-msp430-ti-toolchain | = gcc-msp430-ti-toolchain-9.3.1.2.20210722_1: Success So, trying to place some bounds for comparing to SMT (32 hw threads) and non-SMT (16 hw threads): 33:10:00 SMT -> 39:48:10 non-SMT would be over 6.5 hrs longer for = non-SMT 35:36:51 SMT -> 39:48:10 non-SMT would be over 4 hrs longer for non-SMT As for SMT vs. non-SMT Maximum Observed figures: SMT load averages . . . MaxObs: 360.70, 267.63, 210.84 non-SMT load averages . . . MaxObs: 152.89, 100.94, 76.28 Swap-involved MaxObs figures for SMT (32 hw threads) vs not (16): 173310Mi vs. 33003Mi MaxObsUsed 256332Mi vs. 117221Mi MaxObs(Act+Lndry+SwapUsed) 265551Mi vs. 124776Mi MaxObs(Act+Wir+Lndry+SwapUsed) Just-RAM MaxObs figures for SMT (32 hw threads) vs not (16): 81066Mi vs. 69763Mi MaxObsActive (Given the complications of getting usefully comparable wired figures = for ZFS (ARC): omit.) 94493Mi vs. 94303Mi MaxObs(Act+Wir+Lndry) =3D=3D=3D Mark Millard marklmi at yahoo.com