From nobody Mon Oct 28 01:47:55 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XcGVk4Kp4z5bhCJ for ; Mon, 28 Oct 2024 01:47:58 +0000 (UTC) (envelope-from zagazaw2004@gmail.com) Received: from mail-oo1-xc34.google.com (mail-oo1-xc34.google.com [IPv6:2607:f8b0:4864:20::c34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XcGVk2cR1z4HYB for ; Mon, 28 Oct 2024 01:47:58 +0000 (UTC) (envelope-from zagazaw2004@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oo1-xc34.google.com with SMTP id 006d021491bc7-5ebbed44918so2610571eaf.0 for ; Sun, 27 Oct 2024 18:47:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730080077; x=1730684877; darn=freebsd.org; h=content-language:thread-index:mime-version:message-id:date:subject :in-reply-to:references:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gO0s5O0zfRwl4a5YvJAKYMwMqcLIJFzQPQhntlk27/I=; b=nU7hve4XOvaWRUhzdsyzIngDv1uGwfaCn/bxKhNeY8TiqQxliD6zN2457VRge6+caf 8dUKb9dgRUssW4IaOy4wVyRFH1y+Su4H/qgGYXWnFz1cGifgdME26eJ587+B68uqezTL ok1C0YUPlNmYh82+W563ADww2qirzK1y/nVKEeUM1M06XBcJ8r/xA0M54c3SrZZzCV1F mTAjqTHvXM4lU8SP0JZDK9FvUVRfcbkcvOLJgD/cR2G7wqsNPuX6x698eYjIYUd9565I kRRYFbjxT1nPsyObAAAzXc97NmhYG2A1BArF+rS7c2baHzCGDXroQZe63gnwvofeJW15 LNpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730080077; x=1730684877; h=content-language:thread-index:mime-version:message-id:date:subject :in-reply-to:references:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gO0s5O0zfRwl4a5YvJAKYMwMqcLIJFzQPQhntlk27/I=; b=WSEo4PE3EHGH0YpO/BNMIGDkV95nuTzWU2RNNbufBwnGQnjwSKDWydfnIGRHzjDdFd 5shWburJLakFb7kQZDXFxsYjFGpBJR7ZZ8h8K8x3AUo/C7k2uVwP5ehXxt/KSd6TuEj8 335QW/CFSHzOsGDlGTWnbDpTBPGjrHysqwVcE0znmj7hYDXsAKUjcCpd1dOg+ZUn+yDn V48o2zbNsryLTa60ljqSsSyqK8MPYQ1CORNGagYmB+2ZTvCkKwKB7tIkF2vQZXaTHtZE f8ZhakI2GpMVSPWeubcgH1wsE+HCzJXunKYu560OJHX5L9Cd1LjCpYLwMYXT7BZWMYif hyCw== X-Gm-Message-State: AOJu0YwxrCTqqsJHn0i/0AnXqMinTlO8wCFoN1t5PHql6H0T+tN6J6X5 Qu/jL1m1lNZY1dYlzzmvSH/ZbaIx/Rb+ssRqt+x1NgWs8/PJ6+tgUPgRVLX3ryQ= X-Google-Smtp-Source: AGHT+IGQ3lcrrDvn16Lz1ch6FgUenICugyB2qv5VBYzA6czhPK7nbEc7BPSYpxSRPz63FTRb03S44g== X-Received: by 2002:a05:6820:168e:b0:5e1:d741:6f04 with SMTP id 006d021491bc7-5ec239f68d7mr4021501eaf.3.1730080077051; Sun, 27 Oct 2024 18:47:57 -0700 (PDT) Received: from MICROBOX (c-76-143-116-34.hsd1.tx.comcast.net. [76.143.116.34]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-5ec185d1b8asm1492242eaf.20.2024.10.27.18.47.56 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 27 Oct 2024 18:47:56 -0700 (PDT) From: "Maku Bex" To: "'David Wolfskill'" Cc: References: <012d01db28cc$5fc66ac0$1f534040$@gmail.com> In-Reply-To: Subject: RE: speedup build time Date: Sun, 27 Oct 2024 20:47:55 -0500 Message-ID: <015001db28db$690b03c0$3b210b40$@gmail.com> X-Mailer: Microsoft Outlook 16.0 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Thread-Index: AQJ205h8pLzBefm3/r9z546S2fXHQAHPl2GwsVWNE5A= Content-Language: en-us Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="=-=cRby8em2/lhT9t=-=" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4XcGVk2cR1z4HYB X-Spamd-Bar: ---- This is a multipart message in MIME format. --=-=cRby8em2/lhT9t=-= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Hi David, * What (if any) "max_jobs" (make's "-j" parameter) are you using? 16 * For a 16T system, I expect I would start with a value of around 28. Shoul= dn't this number be equal to $(nproc --all)? * Are you using make's "meta" mode? I do. Yes. * Can you get more memory for that system? Yes, I can bump it to 32GB. This= is a Dell OptiPlex 7080 Micro, and not sure if it is worth upgrading the o= ther components. If your laptop can complete the builds in such a small amount of time, then= there's either something wrong with my machine or the config files. Oh! Th= e build just finished as I was typing this reply and...(not for the weak he= arts): -------------------------------------------------------------- >>> World built in 32437 seconds, ncpu: 16 -------------------------------------------------------------- =20 Here are the contents of: make.conf: CPUTYPE?=3Dicelake-client CFLAGS=3D -O2 -pipe -march=3D${CPUTYPE} #CXXFLAGS=3D${CFLAGS} MACHINE_CPU+=3D avx512 avx2 avx sse42 sse41 ssse3 sse3 f16c MAKE_JOBS_NUMBER=3D 16 BATCH=3Dyes WRKDIRPREFIX=3D/var/ports/tmp CCACHE_DIR=3D/var/ccache/tmp OPTIONS_UNSET+=3D QT5 QT6 CUPS VESA SCFB TWM XCONSOLE XCLOCK GNOME KDE EXAM= PLES \ DEBUG ASPELL NLS DOCS ---------------------------------------------------------------------------= ---- src-env.conf: WITH_META_MODE=3Dyes WITH_CCACHE_BUILD=3Dyes #WITH_DIRDEPS_BUILD=3Dyes <=20 Sent: Sunday, October 27, 2024 19:50 To: Maku Bex Cc: freebsd-current@freebsd.org Subject: Re: speedup build time On Sun, Oct 27, 2024 at 07:00:17PM -0500, Maku Bex wrote: > ... > I'm trying to figure out a way to decrease build time of -CURRENT=20 > and/or -STABLE but it is not working out that well. > I've tried the following: > * Tuning /etc/make.conf and /etc/src-env.conf based on their manpages > (make.conf(5), make(1), src.conf(5)). > * Removed certain options from GENERIC, which reduced the kernel build > time significantly.=20 >=20 > Hardware specs: > * I7-10700T 8C/16T @2.0GHz. > * 16GB RAM. > * 256GB NVMe. >=20 > Currently, 'make buildworld' has been running for 7hrs and counting.=20 > Is there anything else I can do to speed up world build? > .... First, some questions -- whether or not you wish to share the answers, I be= lieve that answering the questions (if only for yourself) may be helpful: * What (if any) "max_jobs" (make's "-j" parameter) are you using? For a 16T system, I expect I would start with a value of around 28. * Are you using make's "meta" mode? I do. * Can you get more memory for that system? Some context: I am in the (definitely peculiar; perhaps unfortunate) habit= of "tracking" FreeBSD head and latest stable branch on a small collection = of machines ... daily. One of those is the laptop that I use for normal "d= esktop" stuff, as well as accessing all other machines on the planet that I= access. (Oh: I also update ports daily, too -- though I let my dedicated = "build machine" handle chromium.) That laptop has 4C/8T & 32 GB RAM. This morning, it took: * 17:46 for stable/14-n269310-bbd018d0aaaf -> stable/14-n269315-b21c677ed28a (0:02 to update /usr/src; 17:44 to build; 0:00 to delete old libraries) * 24:03 for main-n273250-9d585fc395c3 -> main-n273270-cadb71e4b013 (0:03 to update /usr/src; 23:59 to build; 0:01 to delete old libraries) * Updating ports (other than www/chromium) took 1:52. (Those times are "mm:ss" in case it wasn't obvious.) The above is fairly classic "sample size of 1" -- it's not necessarily repr= esentative. Folks who are "sufficiently interested" are welcome to trawl t= hrough the logs at https://www.catwhisker.org/~david/FreeBSD/history/ to ge= t a rough idea of how long it is before the "uname" output from a successfu= l build is logged; there is probably some correlation to build times. I also recently acquired a replacement "build machine": 64C/128T; 256 GB RA= M; a fistful of SATA SSDs (build/boot on one; the rest is a poudriere play = area in a zraid). That's a wee bit faster than the laptop. :-) Peace, david --=20 David H. Wolfskill david@catwhisker.org It has been said that history repeats itself. This is perhaps not quite cor= rect; it merely rhymes. -- Theodor Reik See https://www.catwhisker.org/~david/publickey.gpg for my public key. --=-=cRby8em2/lhT9t=-= Content-Type: application/pgp-signature; name="openpgp-digital-signature.asc" Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iLgEARMKAB0WIQQ3JS+7ltBUFczTX17xqdt536xDxgUCZx7tSgAKCRDxqdt536xD xr6eAgkBRPpdhJ/BhScL8hAI9ytFNXWl9hxw6PFaRY9yZ+Ap8e2Bn2EacYdT4mEF MUpZowUjxk2oqVYBNiLQmRj885VKoYYCCPlmNNOvUw0t3JC4oKntgjEbFVOSlaLx NGhFKgIAk4rX+F03hJY+Mra4ZA0A47L27ZryKFcCQaQf05rRHJi7GxDY =qoPq -----END PGP SIGNATURE----- --=-=cRby8em2/lhT9t=-=--