From nobody Thu Feb 23 19:53:49 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PN3cv5Dd4z3sgL8 for ; Thu, 23 Feb 2023 19:54:07 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-21.consmr.mail.gq1.yahoo.com (sonic310-21.consmr.mail.gq1.yahoo.com [98.137.69.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PN3ct3F6rz4Dnq for ; Thu, 23 Feb 2023 19:54:06 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=WdTrT6P5; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.147 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1677182044; bh=4zpDXKbAlmB1EH2tuMI3l+vN0uoDiQoTJazVUDb7YLw=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=WdTrT6P5k4dMs2haV0OtDBjWq6EzLl6pReJghox+PynpNBmPuTsOJEAI5yuNOvp2zhMNsNd+NrtEG6d8KhGF90Yr0hnRnWqMLFceypfZkctW7JgFwJN2k5JdSiN/+9ktECNAua0d5bShwkSldVYc9g9sRG3ibRyZwGhCq9dnrWfrLI63Ql7xbLxXkUYZUUppYuodPleIFtAID0QPOZNhzvhrGlrUOLiBA7sCo1kVF3CJKApLVFmb5YxwUHWOuKH9Vu5i6Lqt3JVcM4Upqh0OI56QYLjCwXvtPKgurGox31bCXgyq5MlMQW1jCNCxzCsbNIGq2CQGvPO9u32eZnVjsQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1677182044; bh=JUjVklwd91WzX6mRjWOi5KBUPUV6nx3OcshTybH/W2M=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=Q4alG3IVjPFO8giJ5rOcBjE3vBK9/3v+0om3Xa+oSQNeU2XVqWiogYD62dSLPSrIUMJsRGCtIvzT9H9gujTsxN9E0IiyqmJ9vrlNSaxSODqmsk47aKlaXfG3/S79C5yswRczDHgPHAGcz4AjOPr0Km6Le5XXCO/43zspmpxGxFoWAMDJ+1NqK7dViCNb2ElaCSw1jgYBKLEEG4UyOPPVzDOm193BQT7naUunEmyn/pZuuNooFGbvd/KKOQoLSJy6WuwwviprMozlTIQH75kGPTIYJSLipmDtUEwrsaOTI44rqUJyx+xCkd0wQs2CStLZiN4LMXVGhAj6HkymMNdWFg== X-YMail-OSG: mIlcM_sVM1nscbs0txj_UmZPAyMbXv19qZ2C.u4X5foDR6tqui0Hotu8WJGpptE Sf3hlS67kcmL9gRw49sycXWWChGTYmKNtYYSQyfzA7fIHn3tp7aNZ0Nwrblnb5VBdr46WyI5T1Uq 1kTorUmHKWC9Icew29lG6CErzBFO_dfAqMfSuM_bMKEKV6q1ZpI1hB1GT6F3RtS8LCZZQ9qjDgrh 4yz2hQRhARDpyiwwIvMUI.F9qUi_YqxLc4ZGsjezZtvY1kuWawuqsrlZDH5WVwlAmACW_NxJz1.D 6zfgXeT4y9VUEKG.GPkR4vrtKORsKvPxfiqQIWnl02HP09QAOFN9T8Rb6XRco4K.oDM3MN6RCpAe nJwaj3VkxAcAv8cP6qOtwel5OZ3_JCyIiLtbs6sIgDCOf09CRtZxCnzyRX_s2v.M1iJOuLvR_Jmt 4nTOHRZj6.BRYuq2enw4qeIF26wVUDBuv5Kkxq2enVJLnNgF0FmXlu6sc.pZffz_CuwA48HhBggj s9OoTCRbgcnBmV7BowDwdyRmXoLFOQqO9UNli7LycFTRXAW6nBl.ndtWMfQAZLvv9rS7ylQZaWom TSkn1x0jbgQXyiaTfv9g7gpdPup1Mr2IjpcrQry6aL1mEANIyJ8wE7GEcffUxNxGKhD7BQIHWRIQ 5ajiuzRxeZcE0gk26ZHP5X0pRnl1xMT71dS4ogUOwmmdFGqrXdVNQeZul9nMAWDv24wUBDwZAz3P mZwctDKbyjx7w1YTZBw2CISTh3Fo.a5ScQP_DZ8hvzMgHRaIkVejrX__fBXa_GCXh4m2pdG0vbkS sakdi9Y0WVR0sSFhEhh2pZetY0u7sqltc1BNq4z_g4hjO9bO4oY897cWMWO_k24VMvYqYREdeteL VGmsl5jfd6.0wnncKdtYxsElNPF_iDQXD.ZIAYEauWoyfWVeIlLz4.csxw_iASKphmltp4kVz_tM RUEZiNfjwutj2ORUm_qlW2AVeVXqwxEZE2WS9hWJ7524vm3kBFjb2OdXNDXYTk0z6lyu_SuK4x7c m.NCfQSZdOaxTNZdDo0E_9eoCgHJRslrYXvQjKnTsvrVgiqIm1abdktzcAj0Ngu7vMrseq7ZKupD NbYRKB5INrGpc8TitZi3uQQCtuiFnKlys5DKUsrPlx5C2bO25zZ7fGLwbLyQODERbrtTRYck8Vo3 SHi3_diBQSVUIBLJqY9_iUAVbgREIBvn6x4d3kjLLfprGwB1WiqsmJV3xkeT.e_zCvg6v6L7zXr7 Dk5n566wVZhCrKjKvErTM3tUZHF97ezUMUoI_E7jzWSPaPwFpZnILa8CHlM2GCabCBcZoFl3uquO 8uR4bUMzXaI7wN1bjx0KHkR0l06we.ZdPuRiStUlxEWg1WKkv6vpCuqjsZYZvcq0UABnUv9vzGJC XN_M0Zd3o6EiH2iSSYif..N34mo..9cXKLedWZ4Cjef99heqdnV04eyH19u8ytf8LgAmnS0w3fTH a9ecbwshRP3wQFjaaMwZx._u8JhyZedwn7dg.o1MUJm9JPEKmxCrOodHFOIjKgAeaVy.cQQjFt2W xR.AUugyMRDKMWZg7Y_1Rft03uwjcFmFOU5xZ3JfiAsYIbMlewkyOzGXU96ftDwdIFYFpzaPTMMX OT05bL9QOXZ6E3vvRu927gG7JUlZmSG3fq1XQJHdkixy689EGY4rLFzhPiTBXUmSX7vS.XtHchUT 4GvWm0ayvV_SE1veprO_NhzqfhdzugNlg4LVSGQRPt64gsqsZC9h6V2pwjZmxHql6ZeLf7W2BZzq q2Q9.N6QJvA00zVPF8VPrwOJW2LvhJfHseS4G05N5YvTGmlIFEfvQoTLrYsRmQ42QhSKs7mR3TKQ F9BV4CWsm_ed5W65E23mHoAReHBeoMKzvx_BWBqOlVLqlf.EtRu5aImC49skYXG1zomL3hyaAo.2 _lwzZArJYw1ocM6drfxAloah.zC019slrqU.luXA2jCFnKjn2ybdkVeSxc7ZrmTQbzEROGtwpG7J nXju8gM6.SxRK5MkC9V.eLmSMGfF6KZD1O3_kaePJxjdWQ4DiGNC76E8iKjKp8iovViqiUg_x54y XhmeiR30uglQRMm4xgNU.V0hD75xRXCIf52pj5cPUVyLox3kpBWXGoBAjODVqi4OccPftLsfOZgG 1VAR31FjO_RKjnbOgH_J0kCArpQSMBf_nm0CARkmrw0vKu0tYvIbdOLWcecoPfcXvWPwS3D6OWhk FZBedpXoXNLlwUCda5M.OJJ0hgbJtQ5XyzOUnOsr5hNB.Qy29RAeRz59Yc1C83MHCj6cdi_iFGYE CgxM- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Thu, 23 Feb 2023 19:54:04 +0000 Received: by hermes--production-bf1-57c96c66f6-p9vmd (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID c4317a4ce1bf7328f8478cb0b1109b3f; Thu, 23 Feb 2023 19:54:02 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: FYI: Why META_MODE rebuilds so much for building again after installworld (no source changes) [code level bug evidence] From: Mark Millard In-Reply-To: Date: Thu, 23 Feb 2023 11:53:49 -0800 Cc: Bryan Drewery , Current FreeBSD , Peter Content-Transfer-Encoding: quoted-printable Message-Id: <266ED18F-9249-46BB-BF96-1D4C5B46FCFC@yahoo.com> References: <3345EBA5-A09C-4E3F-B94D-39F57F56BDBB@yahoo.com> <73088.1611797582@kaos.jnpr.net> <10819.1677108389@kaos.jnpr.net> <76FA98EF-6184-4D7E-A01F-0EE8117D0D10@yahoo.com> <29887.1677115125@kaos.jnpr.net> <27790339-240F-4C97-97C7-38AFD8DE03D5@yahoo.com> <72419.1677133429@kaos.jnpr.net> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-3.49 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.986]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.147:from]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-current@FreeBSD.org] X-Rspamd-Queue-Id: 4PN3ct3F6rz4Dnq X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N cached_realpath only reports its "cached_realpath:" notice (not the purging one) when it does not find the value via HashTable_FindValue and so does a HashTable_Set : const char * cached_realpath(const char *pathname, char *resolved) { const char *rp; if (pathname =3D=3D NULL || pathname[0] =3D=3D '\0') return NULL; rp =3D HashTable_FindValue(&cached_realpaths, pathname); if (rp !=3D NULL) { /* a hit */ strncpy(resolved, rp, MAXPATHLEN); resolved[MAXPATHLEN - 1] =3D '\0'; return resolved; } rp =3D realpath(pathname, resolved); if (rp !=3D NULL) { HashTable_Set(&cached_realpaths, pathname, = bmake_strdup(rp)); DEBUG2(DIR, "cached_realpath: %s -> %s\n", pathname, = rp); return resolved; } /* should we negative-cache? */ return NULL; } cached_realpaths is global: static HashTable cached_realpaths; So with -ddM why do I see lots of "cached_realpath:" notices for the same path? For example: # grep "tmp/legacy/usr/sbin/ln\>" = /usr/obj/BUILDs/main-amd64-nodbg-clang/sys-typescripts/typescript-make-amd= 64-nodbg-clang-amd64-host-2023-02-23:10:20:26 | more cached_realpath: = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln -> = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /bin/ln cached_realpath: = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln -> = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /bin/ln Caching 02:49:37 Feb 23, 2023 for = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/usr.bin/aw= k/awkgram.tab.h.meta: 22: file = '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac= y/usr/sbin/ln' is newer than the target... cached_realpath: = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln -> = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /bin/ln Caching 02:49:37 Feb 23, 2023 for = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln cached_realpath: = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln -> = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /bin/ln Caching 02:49:37 Feb 23, 2023 for = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln cached_realpath: = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln -> = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /bin/ln Caching 02:49:37 Feb 23, 2023 for = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln cached_realpath: = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln -> = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /bin/ln Caching 02:49:37 Feb 23, 2023 for = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln cached_realpath: = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln -> = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /bin/ln Caching 02:49:37 Feb 23, 2023 for = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/usr.bin/aw= k/awkgram.tab.h.meta: 22: file = '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac= y/usr/sbin/ln' is newer than the target... . . . A possible cause is something I ran into while looking around: /* A read-only range of a character array, NOT null-terminated. */ typedef struct Substring { const char *start; const char *end; } Substring; . . . MAKE_STATIC Substring Substring_Init(const char *start, const char *end) { Substring sub; sub.start =3D start; sub.end =3D end; return sub; } . . . /* Find the entry corresponding to the key, or return NULL. */ HashEntry * HashTable_FindEntry(HashTable *t, const char *key) { const char *keyEnd; unsigned int h =3D Hash_String(key, &keyEnd); return HashTable_Find(t, Substring_Init(key, keyEnd), h); } . . . /* A read-only range of a character array, NOT null-terminated. */ typedef struct Substring { const char *start; const char *end; } Substring; . . . MAKE_STATIC Substring Substring_Init(const char *start, const char *end) { Substring sub; sub.start =3D start; sub.end =3D end; return sub; } . . . /* Find the entry corresponding to the key, or return NULL. */ HashEntry * HashTable_FindEntry(HashTable *t, const char *key) { const char *keyEnd; unsigned int h =3D Hash_String(key, &keyEnd); return HashTable_Find(t, Substring_Init(key, keyEnd), h); } . . . /* This hash function matches Gosling's Emacs and java.lang.String. */ static unsigned int Hash_String(const char *key, const char **out_keyEnd) { unsigned int h; const char *p; h =3D 0; for (p =3D key; *p !=3D '\0'; p++) h =3D 31 * h + (unsigned char)*p; =20 *out_keyEnd =3D p; return h; } But after the loop: *p=3D=3D'\0' so *out_keyEnd=3D=3D'\0' and the FindEntry Substring_Init(key, keyEnd) ends up including the '\0' byte. But note that the h in Hash_String did not include the '\0' byte. Call this h value h_VALUE0 for later reference. Then look at: /* This hash function matches Gosling's Emacs and java.lang.String. */ unsigned int Hash_Substring(Substring key) { unsigned int h; const char *p; =20 h =3D 0; for (p =3D key.start; p !=3D key.end; p++) h =3D 31 * h + (unsigned char)*p; return h; } This h does include the '\0' byte so h=3D=3D(unsigned int)(31*h_VALUE0). I expect the mismatched hash values explain the repeated "cached_realpath:" notices for the same path: inserted but never found. =3D=3D=3D Mark Millard marklmi at yahoo.com