From nobody Wed Aug 23 17:01:25 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RWCD530Cgz4r6VW; Wed, 23 Aug 2023 17:01:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RWCD52MZ1z4N0s; Wed, 23 Aug 2023 17:01:25 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692810085; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=RW1gNiJfCTNwifqoM0SezB8XCBgToYCwi9HNDnbuD+E=; b=ixMXcrUYTJ0SokW+/DC9ZLYhWuiNT0avUAdOvU06Y7fcxGO3eza3Dwc2mLAy0c7OiYXo2n 1Fl0forKSxOl+dnrKlhTk5+OdNDjwzS6+Gme1ou2pa+dHGBAZz+QaZMdKQjACTke+A2oGc EgH5jJLaflx45bdYM1A5NvgWHPkzousIKPHqIR/rCKDtevvmdPAPLCwmfTjCvbnGSJYO20 WXfnPNCBPWGOaOB9aTTK+jMglwGvLb6oLowi1dNdzaszTXdKsNm1vIiuHe4bV+715GX+WJ awfHGkYSyoyOktO/WW6mrZanpDAi480jI/oHaVV4pRGDG7VORdD8ZzCvBHrN1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692810085; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=RW1gNiJfCTNwifqoM0SezB8XCBgToYCwi9HNDnbuD+E=; b=v08jtygF4kdpsJWg4J1SNIMTlAZQZopo1q8RLpKBSqOK7nxcACk9lzkI/ve5iWxQNMX+hJ oiLpfWv5s5GeK8nLVZZ1Gz5FJgo8orb5ZfBoOPTIGT8Cux32yK+CCf/eskcRkRS9D7juE4 75ELtlQbC+3QNML8MT4465gDBQyIvTJQidNRJC60gjhuS1LJhdMelC/LabCCpgxTULtlNJ rBOgBbb3lQ2XZQvRFhvNer5G9kKkufmGcLjKUGhJEEUdbqxjv0q+ZeTEfw/VVJ20TKN5Pn de2oZR4lH1Q7lc77jOccVIcteGDJCP08KcSb8pAzqjBw6wfv9YhSp2XpBB20jA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692810085; a=rsa-sha256; cv=none; b=fIxANSwe+AM4xe9I8PzABUNDoBElpsqUrOLn5fL1dQgQOswE/IOO9tYMJBWrsPk6Bhuyxe Mlw4Xbx0zC7JCbnZJkDcX2hdrHfm47qEI1IPUy9R40RmL2fMKryPNXnr7+rHXNWtavwFwK e3vzyaYEzovsS697XinpqkUt7VYhRbLVDPFwPiD9UVuqpfPLBT2WMLyOffD3yAmQxq6IEd Al5HZ1G7gSd7+XECBU1PaUHY17sxr3aIXps3fqTq/2kHQjPObQIYXBdfIvb1NXBWUguAB/ QdPATKDhpi9HXVYMDw2Yl8SKtKsnb+2IvxX752nOUpyC2LTl89iz6CkykKTljw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4RWCD51R6QzshP; Wed, 23 Aug 2023 17:01:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 37NH1Pf9070016; Wed, 23 Aug 2023 17:01:25 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 37NH1PJF070013; Wed, 23 Aug 2023 17:01:25 GMT (envelope-from git) Date: Wed, 23 Aug 2023 17:01:25 GMT Message-Id: <202308231701.37NH1PJF070013@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Jessica Clarke Subject: git: 69cfdc81ea7b - main - tools/build/make.py: Keep bootstrapped bmake binary up-to-date List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jrtc27 X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 69cfdc81ea7bdfe1e7300a7e1106419156aee434 Auto-Submitted: auto-generated The branch main has been updated by jrtc27: URL: https://cgit.FreeBSD.org/src/commit/?id=69cfdc81ea7bdfe1e7300a7e1106419156aee434 commit 69cfdc81ea7bdfe1e7300a7e1106419156aee434 Author: Jessica Clarke AuthorDate: 2023-08-23 16:56:50 +0000 Commit: Jessica Clarke CommitDate: 2023-08-23 16:56:50 +0000 tools/build/make.py: Keep bootstrapped bmake binary up-to-date We currently assume that any existing bootstrapped bmake binary will work, but this means it never gets updated as contrib/bmake is, and similarly we won't rebuild it as and when the configure arguments given to boot-strap change. Whilst the former isn't necessarily a huge problem given WANT_MAKE_VERSION rarely gets bumped in Makefile, having fewer variables is a good thing, and so it's easiest if we just always keep it up-to-date rather than trying to do something similar to what's already in Makefile (which may or may not be accurate, given updating FreeBSD gives you an updated bmake, but nothing does so for our bootstrapped bmake on non-FreeBSD). The latter is more problematic, though, and the next commit will be changing this configuration. We thus now add in two checks. The first is to compare MAKE_VERSION against _MAKE_VERSION from contrib/bmake/VERSION. The second is to record at bootstrap time the exact configuration used, and compare that against what we would bootstrap with. Reviewed by: arichardson, sjg Differential Revision: https://reviews.freebsd.org/D41556 --- tools/build/make.py | 58 ++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 49 insertions(+), 9 deletions(-) diff --git a/tools/build/make.py b/tools/build/make.py index 74454aa51b08..e25922c9a70d 100755 --- a/tools/build/make.py +++ b/tools/build/make.py @@ -72,31 +72,71 @@ def run(cmd, **kwargs): subprocess.check_call(cmd, **kwargs) +# Always bootstraps in order to control bmake's config to ensure compatibility def bootstrap_bmake(source_root, objdir_prefix): bmake_source_dir = source_root / "contrib/bmake" bmake_build_dir = objdir_prefix / "bmake-build" bmake_install_dir = objdir_prefix / "bmake-install" bmake_binary = bmake_install_dir / "bin/bmake" + bmake_config = bmake_install_dir / ".make-py-config" - if (bmake_install_dir / "bin/bmake").exists(): + bmake_source_version = subprocess.run([ + "sh", "-c", ". \"$0\"/VERSION; echo $_MAKE_VERSION", + bmake_source_dir], capture_output=True).stdout.strip() + try: + bmake_source_version = int(bmake_source_version) + except ValueError: + sys.exit("Invalid source bmake version '" + bmake_source_version + "'") + + bmake_installed_version = 0 + if bmake_binary.exists(): + bmake_installed_version = subprocess.run([ + bmake_binary, "-r", "-f", "/dev/null", "-V", "MAKE_VERSION"], + capture_output=True).stdout.strip() + try: + bmake_installed_version = int(bmake_installed_version.strip()) + except ValueError: + print("Invalid installed bmake version '" + + bmake_installed_version + "', treating as not present") + + configure_args = [ + "--with-default-sys-path=" + str(bmake_install_dir / "share/mk"), + "--with-machine=amd64", # TODO? "--with-machine-arch=amd64", + "--without-filemon", "--prefix=" + str(bmake_install_dir)] + + configure_args_str = ' '.join([shlex.quote(x) for x in configure_args]) + if bmake_config.exists(): + last_configure_args_str = bmake_config.read_text() + else: + last_configure_args_str = "" + + debug("Source bmake version: " + str(bmake_source_version)) + debug("Installed bmake version: " + str(bmake_installed_version)) + debug("Configure args: " + configure_args_str) + debug("Last configure args: " + last_configure_args_str) + + if bmake_installed_version == bmake_source_version and \ + configure_args_str == last_configure_args_str: return bmake_binary + print("Bootstrapping bmake...") - # TODO: check if the host system bmake is new enough and use that instead - if not bmake_build_dir.exists(): - os.makedirs(str(bmake_build_dir)) + if bmake_build_dir.exists(): + shutil.rmtree(str(bmake_build_dir)) + if bmake_install_dir.exists(): + shutil.rmtree(str(bmake_install_dir)) + + os.makedirs(str(bmake_build_dir)) + env = os.environ.copy() global new_env_vars env.update(new_env_vars) - configure_args = [ - "--with-default-sys-path=" + str(bmake_install_dir / "share/mk"), - "--with-machine=amd64", # TODO? "--with-machine-arch=amd64", - "--without-filemon", "--prefix=" + str(bmake_install_dir)] run(["sh", bmake_source_dir / "boot-strap"] + configure_args, cwd=str(bmake_build_dir), env=env) - run(["sh", bmake_source_dir / "boot-strap", "op=install"] + configure_args, cwd=str(bmake_build_dir)) + bmake_config.write_text(configure_args_str) + print("Finished bootstrapping bmake...") return bmake_binary