From nobody Thu Feb 23 20:25:03 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PN4LC6dPGz3sjrZ for ; Thu, 23 Feb 2023 20:26:27 +0000 (UTC) (envelope-from sjg@juniper.net) Received: from mx0a-00273201.pphosted.com (mx0a-00273201.pphosted.com [208.84.65.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.pphosted.com", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PN4LC5Ltjz4Mn4; Thu, 23 Feb 2023 20:26:27 +0000 (UTC) (envelope-from sjg@juniper.net) Authentication-Results: mx1.freebsd.org; none Received: from pps.filterd (m0108156.ppops.net [127.0.0.1]) by mx0a-00273201.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 31NBt98m006042; Thu, 23 Feb 2023 12:26:06 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; h=to : cc : subject : in-reply-to : references : from : mime-version : content-type : content-id : content-transfer-encoding : date : message-id; s=PPS1017; bh=zOhYNp0S09uoEra7w6m81jlDweaERvlIpUILMC/jQHk=; b=QJyIvV2mTJI3nDyRyWgKpqs+uYTSFvWNi/xBMC4/lAPw6YyHRRhHviODJ80X8G3KOR2a S15p7iD0JVYYzMq4DWl9ZfCaT4yaITx47fsBGz1zpJfZxJ3OAuQPaNfjIiSNfvucQIC5 swl+KGjSrC9PVtHoqarIb8AQ9CxeC8E2IAcUOthl9bjW6PbcOeqrsshxNnkqjKfzeosb j15bawvvlhYU2uYHZjLa3XwEP2Nv5hng3H4bCsRBW1ucpwoB0R7kbZQ9aAld/GlJR7Tb zZvuhg1Zmi/gBBAr1E1DaAE5MP+ZEUxnYkET95X33foTMuHxyRbQG6DcYHoOUsFZ7sfz OA== Received: from dm4pr02cu001-vft-obe.outbound.protection.outlook.com (mail-centralusazlp17012028.outbound.protection.outlook.com [40.93.13.28]) by mx0a-00273201.pphosted.com (PPS) with ESMTPS id 3nwyb99mc6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 23 Feb 2023 12:26:06 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=El1GUp3vRq+mR8oJm8s6eKjMkllcabb1+h2/39h6oAw0G9UiXM3lTZkgCJT5c5CNOtDvvyibOoqLtJIO7GEKYZw6wO5kQVYbNRKypzv3hc2Pu6h81XTU7fQ/LNsA/9aP+DPnu50oMz2uOcjsZANeEfm0sLn2vHxqWZsf/hd0lMDpGqyo89lzpl92QQNcRSd1iGqt4yGnj4x6wfxZ4SlNKJWseFd0i+Chzou2KZFmNdxhgq748mMk/4Ol3wDLDpWNRSlwwhS68kjv4SbipY0eFzuWakFa4vlm7FTgXA82+zAhntG9vz7enSDLi5fj6K117WNpwX2RBkGltCX9vwS0Ng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zOhYNp0S09uoEra7w6m81jlDweaERvlIpUILMC/jQHk=; b=cFmZR2ZuIqHE+BkcFD9d569Jx8jd108uPXwH9fTHFa6op3Pdqcr1cd3yHIjSyKFawhj/pzhY3mv3CMH/W/nIcP21cVnIg/NORlLWBldf2XH/YtM86/J6yZ8eDjfroXM+Qx87DSPny8+dusiUXCtGbFh5tLB+zeo2DMz8yMJI41gMAR2O/JstPO4fS8MJWoV2f72F3RuKLXPni8Jac3oCKw8sMhU5RTcqbEEW9xnNdz2QDosbKyNouR5iVCA40cWy9illpI4PcZTHtFBz6jPUaAhei0di2rtlUWSmJmOS2Rp72WFdjim8BoOoEWlhLrVEHFmc7hteaEzOFzxfqrHg9g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=softfail (sender ip is 66.129.239.15) smtp.rcpttodomain=yahoo.com smtp.mailfrom=juniper.net; dmarc=fail (p=reject sp=reject pct=100) action=oreject header.from=juniper.net; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zOhYNp0S09uoEra7w6m81jlDweaERvlIpUILMC/jQHk=; b=M6o2DcuJm4HRF0JGp1J4sqd3Llw3cnnoxHl9fG0Mvws1uOzgquwcprwPtHVihfkBaooG0LQpKl8R0kKWcItWJa4A/8T3II9KfFf2/zoPzbGfMuOcd/tF4Z9DKUSPc8zSq92oAkWaIAjWcDf1hNcsc2Ah9n0C/0aHBm4IIXqCZPU= Received: from MW4PR03CA0014.namprd03.prod.outlook.com (2603:10b6:303:8f::19) by SN6PR05MB5950.namprd05.prod.outlook.com (2603:10b6:805:fb::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6111.19; Thu, 23 Feb 2023 20:26:04 +0000 Received: from MW2NAM12FT065.eop-nam12.prod.protection.outlook.com (2603:10b6:303:8f:cafe::9c) by MW4PR03CA0014.outlook.office365.com (2603:10b6:303:8f::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6134.21 via Frontend Transport; Thu, 23 Feb 2023 20:26:03 +0000 X-MS-Exchange-Authentication-Results: spf=softfail (sender IP is 66.129.239.15) smtp.mailfrom=juniper.net; dkim=none (message not signed) header.d=none;dmarc=fail action=oreject header.from=juniper.net; Received-SPF: SoftFail (protection.outlook.com: domain of transitioning juniper.net discourages use of 66.129.239.15 as permitted sender) Received: from p-exchfe-eqx-02.jnpr.net (66.129.239.15) by MW2NAM12FT065.mail.protection.outlook.com (10.13.180.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6156.7 via Frontend Transport; Thu, 23 Feb 2023 20:26:03 +0000 Received: from p-exchbe-eqx-01.jnpr.net (10.104.9.14) by p-exchfe-eqx-02.jnpr.net (10.104.9.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.30; Thu, 23 Feb 2023 14:26:03 -0600 Received: from p-mailhub01.juniper.net (10.104.20.6) by p-exchbe-eqx-01.jnpr.net (10.104.9.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.30 via Frontend Transport; Thu, 23 Feb 2023 14:26:03 -0600 Received: from kaos.jnpr.net (kaos.jnpr.net [172.23.255.201]) by p-mailhub01.juniper.net (8.14.4/8.11.3) with ESMTP id 31NKQ2WW015535; Thu, 23 Feb 2023 12:26:02 -0800 (envelope-from sjg@juniper.net) Received: by kaos.jnpr.net (Postfix, from userid 1377) id 98F7820E6B; Thu, 23 Feb 2023 12:25:03 -0800 (PST) Received: from kaos.jnpr.net (localhost [127.0.0.1]) by kaos.jnpr.net (Postfix) with ESMTP id 9879D20EE9; Thu, 23 Feb 2023 12:25:03 -0800 (PST) To: Mark Millard CC: Bryan Drewery , Current FreeBSD , Peter , Subject: Re: FYI: Why META_MODE rebuilds so much for building again after installworld (no source changes) In-Reply-To: References: <3345EBA5-A09C-4E3F-B94D-39F57F56BDBB@yahoo.com> <73088.1611797582@kaos.jnpr.net> <10819.1677108389@kaos.jnpr.net> <76FA98EF-6184-4D7E-A01F-0EE8117D0D10@yahoo.com> <29887.1677115125@kaos.jnpr.net> <27790339-240F-4C97-97C7-38AFD8DE03D5@yahoo.com> <72419.1677133429@kaos.jnpr.net> Comments: In-reply-to: Mark Millard message dated "Thu, 23 Feb 2023 00:07:54 -0800." From: "Simon J. Gerraty" X-Mailer: MH-E 8.6+git; nmh 1.7.1; GNU Emacs 28.1 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <97974.1677183903.1@kaos.jnpr.net> Content-Transfer-Encoding: quoted-printable Date: Thu, 23 Feb 2023 12:25:03 -0800 Message-ID: <30.1677183903@kaos.jnpr.net> X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MW2NAM12FT065:EE_|SN6PR05MB5950:EE_ X-MS-Office365-Filtering-Correlation-Id: bc20adc2-36c3-4259-46a6-08db15dc2f98 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: mQiHWCLdGBa7XwKHqn6gR4MjVBTQ37MqHHAHxt4tjkMqaYpYdnDPNB5NKX9xvQ8K44WUmvhK+MLZDot+xFbAxVuSEcTqAUMjtSjXncQEFOrMDBEEGJZFSG7dT4CIvyrh6ao6MoIM+DuxXR8czmqc+kKfk6Oug5XobhImzuepQ5bq3PDBAHKxf8pcmhyc5IYe9cdMvw9j8FSLQZPhRisej0hJHKv5thfEPwRk2ceTZtaTk49zqvMPO4fgEvQ9y8C+RD1LZNwsd9B8rWlDW+lBtEkxGmiXATd3MBTguSB7vHh1rVvEIToc1I2ThAJrwMA+z96s4nPW8acnpr0VmZ12OksRVohrPv6PGrQSa57EyBUokQcGUyfXnemMK/BpM0s0eAaA8FPb0x8d7DeZd2GbrzcikPHwA89rnOgwp+Wl0J12nuUkjjqm/XBLZv8Ux7NFAYiyZNKhHLyb0+kbpQvW4VpvDqiJl2k4hgcjshCFpBPRY8FR7OV39Z9zk8zC0abGdckWs3DybaF/7vjP0DAoXcu5lnQT9MjGzlkGz2CxYAcT61SC97N6EXIJfxp8vor22anIKvBtWw7/6nK4Sv7A2SNrPwHsxeiDJQXHtmIy9odN9fSBxnEVOABIeoTZMJztNfPwZzQQXI+pnR2P1eIK1eHFnLDbvamO98TMvam95z9Cz9DAmsj51tTXurpCdY2Q3D6m8seagogJW5+nq5dH+fiXbbI/5JHDCXFFvgXFig4= X-Forefront-Antispam-Report: CIP:66.129.239.15;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:p-exchfe-eqx-02.jnpr.net;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230025)(4636009)(39860400002)(136003)(376002)(346002)(396003)(451199018)(40470700004)(36840700001)(46966006)(4326008)(336012)(70586007)(316002)(8676002)(40460700003)(7696005)(54906003)(41300700001)(356005)(8936002)(70206006)(6916009)(55016003)(86362001)(5660300002)(478600001)(82310400005)(6266002)(186003)(26005)(107886003)(9686003)(47076005)(7126003)(40480700001)(2906002)(82740400003)(81166007)(83380400001)(36860700001)(36900700001);DIR:OUT;SFP:1102; X-OriginatorOrg: juniper.net X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2023 20:26:03.7148 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bc20adc2-36c3-4259-46a6-08db15dc2f98 X-MS-Exchange-CrossTenant-Id: bea78b3c-4cdb-4130-854a-1d193232e5f4 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=bea78b3c-4cdb-4130-854a-1d193232e5f4;Ip=[66.129.239.15];Helo=[p-exchfe-eqx-02.jnpr.net] X-MS-Exchange-CrossTenant-AuthSource: MW2NAM12FT065.eop-nam12.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR05MB5950 X-Proofpoint-ORIG-GUID: eOBP4ChjHLSAoaaqYEzY6BibafU8p9pq X-Proofpoint-GUID: eOBP4ChjHLSAoaaqYEzY6BibafU8p9pq X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.170.22 definitions=2023-02-23_13,2023-02-23_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=outbound_spam_notspam policy=outbound_spam score=0 spamscore=0 malwarescore=0 mlxlogscore=999 impostorscore=0 mlxscore=0 adultscore=0 priorityscore=1501 suspectscore=0 lowpriorityscore=0 clxscore=1015 phishscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000 definitions=main-2302230169 X-Rspamd-Queue-Id: 4PN4LC5Ltjz4Mn4 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:26211, ipnet:208.84.65.0/24, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N Mark Millard wrote: > >> Also, OBJTOP is not constant over all the parts of > >> buildworld buildkernel . Having the late-substitution > >> form of notation ${OBJTOP} might not be appropriate > >> for the content of .MAKE.META.IGNORE_PATHS . > > > > Fwiw .MAKE.META.IGNORE_PATHS is evaluated when meta_init() is > > called after all the makefiles have been read - and had a > > chance to influence .MAKE.MODE, so I'm not sure how varaiablity of > > OBJTOP would matter? > = > To my knowledge, there is no place to find documentation > about when/how-often the .MAKE.META.IGNORE_PATHS original > text is reevaluated other than the above statement or > source code inspection. (Others have a similar status.) True. That should be fixed. .MAKE.META.IGNORE_PATHS is only evaluated once by make after all makefiles have been read. It is use to populate a list. By contrast .MAKE.META.IGNORE_PATTERNS and .MAKE.META.IGNORE_FILTER are evaluated for every line of filemon output - but again all that happens after all makefiles have been read. > -dV -V.MAKE.META.IGNORE_PATHS use does list ${__MAKE_SHELL} > but lists /bin/sh without the -dV . (So -V use does not > give a direct clue at what you report.) One of the local changes to bmake in FreeBSD is that = without -dV -V shows the fully resolved value. > I got past the issue using :=3D before reading the above. > (I'm also using MAKEOBJDIR instead of OBJTOP currently.) Per my last response, I'd be pretty sure MAKEOBJDIR is incorrect. > = > >>> .MAKE.META.IGNORE_PATHS+=3D ${OBJTOP}/tmp/ > >> > >> (Ignoring the variability of OBJTOP issue . . .) > >> > >> I do not expect that would work: ignoring things > >> it likely should not. > > > > Sure, but it may be useful as an experiment to ensure things are > > behaving as expected. > = > As a test: > = > .if ${.MAKE.LEVEL} =3D=3D 0 > .MAKE.META.IGNORE_PATHS+=3D ${MAKEOBJDIR:tA}/tmp/ > .MAKE.META.IGNORE_PATHS:=3D ${.MAKE.META.IGNORE_PATHS} > .endif Lose the .if ${.MAKE.LEVEL} =3D=3D 0 it is almost certainly keeping things from working as expected. > I still get things like: > = > /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENE= RIC-NODBG/modules/usr/main-src/sys/modules/zlib/x86.meta: 23: file '/usr/o= bj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy/usr/s= bin/realpath' is newer than the target... > Building /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64= /sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/zlib/x86 Because that will not be level 0 and so .MAKE.META.IGNORE_PATHS is not set. > = > and: > = > /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENE= RIC-NODBG/modules/usr/main-src/sys/modules/xl/opt_platform.h.meta: 12: fil= e '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/leg= acy/usr/sbin/ln' is newer than the target... > Building /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64= /sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/xl/opt_platform.h > = > for both of a pair of back-to-back runs of buildworld buildkernel. > = > FYI: The file system is zfs with mounts that look > like: > = > zoptb /zoptb > zoptb/BUILDs /usr/obj/BUILDs > . . . > zoptb/BUILDs/main-amd64-nodbg-clang /usr/obj/BUILDs/main-amd64-nodbg-cl= ang > . . . > zoptb/ROOT/main-amd64 / > . . . > zoptb/tmp /tmp > . . . > = > # bectl list > BE Active Mountpoint Space Created > 13S-amd64 - - 4.97G 2021-08-20 16:57 > 13_0R-amd64 - - 4.30G 2021-08-20 16:56 > 13_1R-amd64 - - 4.12G 2022-03-10 12:38 > main-amd64 NR / 7.42G 2023-02-19 15:37 > old-main-amd64 - - 2.25G 2023-02-09 19:07 > = > (I use zfs in order to use bectl on a couple of > systems, not for redundancy.) > = > = > >> Also, I'd rather grow a smaller set of ignores > >> gradually to make it easier to detect if an > >> addition starts causing a problem and can be > >> backed out. Starting with everything ignored > >> would make things much harder to figure out > >> when ignoring creates a problem. > > > > Yes. > > > >> > >>> You might need ${OBJTOP:tA}/tmp/ > >>> or both. > > > > I found it necessary in the unit tests to add :tA to both TMPDIR > > and .OBJDIR to get sane result on one test platform. > > > >>>> It is using paths that match the -dM output lines ( sbin > >>>> use despite sbin -> ../bin being a symbolic link). > > > > use :tA if you want to ensure consistent results. > = > So, for each: > = > .MAKE.META.IGNORE_PATHS+=3D ${MAKEOBJDIR}/tmp/legacy/usr/sbin/${ignore_l= egacy_tool} > = > I need to form an overall :tA on the path? Something > like: > = > .if ${.MAKE.LEVEL} =3D=3D 0 I think you need to first get rid of that level 0 check before worrying about anything else. > .for ignore_legacy_tool in awk cap_mkdb cat cp crunchgen crunchide dd eg= rep env file2c gencat grep gzip jot lex lb ln m4 mkcsmapper mktemp mv patc= h realpath rm sed sh touch truncate uudecode uuencode > xargs > IGNORELEGACY_${ignore_legacy_tool}=3D ${MAKEOBJDIR}/tmp/legacy/usr/sbin/= ${ignore_legacy_tool} > .MAKE.META.IGNORE_PATHS+=3D ${IGNORELEGACY_${ignore_legacy_tool}:tA} > .endfor > .for ignore_other_tool in ctfconvert objcopy nm > IGNOREOTHER_${ignore_other_tool}=3D ${MAKEOBJDIR}/tmp/usr/bin/${ignore_o= ther_tool} > .MAKE.META.IGNORE_PATHS+=3D ${IGNOREOTHER_${ignore_other_tool}:tA} > .endfor > .MAKE.META.IGNORE_PATHS:=3D ${.MAKE.META.IGNORE_PATHS} > .endif > = > Such seems to make no difference to the text reported via > -dV -V.MAKE.META.IGNORE_PATHS in my context. Yes, right now I think your main problem is only setting .MAKE.META.IGNORE_PATHS at level 0