From nobody Thu Feb 23 20:23:02 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PN4Gg62QSz3sjKv for ; Thu, 23 Feb 2023 20:23:23 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic313-20.consmr.mail.gq1.yahoo.com (sonic313-20.consmr.mail.gq1.yahoo.com [98.137.65.83]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PN4Gf3qjMz4LnB for ; Thu, 23 Feb 2023 20:23:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=DLkWpTQI; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.83 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1677183800; bh=rBnguEV2SrfS/hPvkNE9wl+XV0JSkNCu55cAXP1Qw6o=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=DLkWpTQIEOyiSMY6FvK21G1JIcfUBPYgqiM0038MmgZr2PfDAXkcQqHTh4nf5binnQYFJFSSHUuwG1ZuNUVpTWmyxm6h4uZ6EpEQPP8kM/Y6meMHrDRZDA3I87bxeGa3NvsLz+agAvZ2hGuM0rKuGFmoe/U+F3fnNZxI+Sd+0nIP2786Kx8MzVbvk7o71vVF66sdGYd9GWWhyd6S++jhA7dRh78eHMTa5ii8ggb7jtVwX6wn5gAQgrztUDJc95ob2y8BjAd1EZuDgrmZBvLY6zWiCdesEaWWNxEus8He/NRtLlbEbtW9rtnKAz0B4FzJekSYrX+wy8+W89yl9GxN6A== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1677183800; bh=KSnBRjHIEUxsRsGl8z49Mfi6eBlYGO35lOJKXXABQhR=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=E4j/vAkenE0224rHsdekPeCYOBgKxGDQilNC65LJCIPL0lVHIB0fWsxn4Jrp+ESlrW8/UiyuuRkYuSgqsbsldc/wr76AOf5MAasKi1fBXJm87qy151G5w+mZ0dp0FxEf5NYlvxmJrkbCk6N976RDcwhNlAN/qTYfTkBpUzAlvrLOlKj8+iSojR/iDd+y8Ir3SAgfRoVy9RpZ92pII7C/t8twiJDvpcjS7BFudVj3lDy57pimRgEgvny/oLL1E6+8tn9lnWrU04bKb5u3Wz9bEFiLTy3RlIHDbEa7rRQzhykqcICfzu3uFdp5E6cjewqBkZhdTkmgh3eJtCTntTjwsg== X-YMail-OSG: tb3CyAYVM1kxel3qJY4EM7LMFRSTr.cJJYkjMkiWSRyXp4RNPxVW3kuhvl_x5cT QMSprd8TkifVjnyk5pi5qdQIdbah61hyq_9eJ7Lrqjv6MfE6bLaCy39JvUIv0Ch_oH.gzzcM4MeO ivUZtfJQL4c4_zMQ30peXIT.vEU.zzngJmp4MOGoOJBVPVPkLCm7MQxSWQL6ozmsHtf32nARdzho buv8LGzjkd0DlHdBkaHgJkXmPa0JFW9vxj8l9XmMc2EH0S5n919Bviv32BW2_Zfc4KNavTAYYrpy vUyKsM5Xi8n7sKqwXk1jqlG58_PyQCgdJsFTguUoH0_k2BJkOHn8doV8Ro4aZD4XiV.hNqDWJGjQ ZPwmFey6AG2q3gjJ5UyKUqoaht.p0jM8NIqACh5LGJfvoqipgHUwq_T0vD3zDwf0MntBn4yXYHzj syDLFB1Mtfvxh4avtuUxKxVsAqh.n99Z0HVJ0UBWP2UVI_s3SoVG.3dg8GJmyDRmKS1GD5bb6cwE GOwyodMLya6exxb3GJesvUmcYxLC8Ik_.Ou98VOu2HfYk_sEMH9y68v_6lPFIpiJaoVM27B4ZIkr 4xXcMC0MSQ6zWrDU2xuJwkA1bK0xDkt6lakrtZIpdsubvCC3J6tM4PNFgtzm2QR19SquueuQOxSc PrtEsbfihZRCJC42QB.P0D0VEPP7pSwb5QbCVEhnqpCpLZR0HTN2mogJM7RgN0jbdULmsfd6OI3I 1foVBFrOz9Uob_iUeu0lSMmMdlqqS4XXm1UU9Z3mglOHQEkeEm8Jal.NUEj6NaO9KoXBx8IdaMy8 0uZa6Imp6pT8_UJDG6Ui8crRYjribahf2tUF6XTfnoTrNN2.YWfpBDjEhX9BfCiMXpXcuuzvTBtP aMKXlb_aS7DzUfHdmDHC4CZkoyluxhISNxYvYppXAsWXkMkd0Dl03Eav3xBHVUP4X3uYQsoa4byb gn.HGQJmZboR6X7FeZeD.GaGvroDBZn_fGpIfVSVvNYRmvgUxjzIHKqsgEMym4YJh76vmwrCbztv kkRJ1sGIZyKRAkIBSe1uD83AUBJNyuBFk4SEJRcDC96t0vjtLOjplNTVUekSvss33IukrIl7WHJn OGKBaPyDknhORAd4TUEHiLGyXSs2jpF8.KHVD5_YLIJQHx7Ni7hR__w61zDlMDbvypk1M9wXd1xR mat_OzWBMYR_myMeb5Wes0wZByXS8BAhFWrN6hG0Uja55R8sMX3uU2s8MXUl2F.w.K6UAvpqY1e0 FKC.TPce3V.SyTa0Z3xo.5fEc5TbrAycWr51JJnzDNpebCcscR2PlM6XEcZklfYmPZoEi6DHVQ6g wMmhA0r5QpQAnwSMAjsRQAJrQNBMYkDgQhF1wnBxO_tKYNUh4TJOq1lyU8fI2OS0NuRprhv1byVn B_QnOfHAe83Dixtxx_JybOXpZAgF9Ac9eFmcouhxHpTrWxlxCxT5vHz58Kl8fatSvfFo8q3nMBwz mM8yfsdqHSgc._Vza_bA2EkAmZqLvWhAcHm6tyNvawjGH9TXmDmn8NVxvd2BcUan6HDDIypjOeEK skh5.3Xz_e.crX4rMkQjU2JqoKB.Ztm7.9xHr.C7hALSypXq.x2FIKsc2Q5W4RVRgCJTgDQ5Q6bA GJwVEoITwtLPyEpY3Rc62nqxOvWHGxA0x4k2u.i6_62_cna89kf.E6ASEpwr9yzMY_Oj7KmzKG2h YzZOZkq63rXlNWAGyJH90rmIeM21A_Y0y4siy1tGG4LNbW8AVUE0u0BKTRGrK1ZjgEmQTPvl98YV I_V2CLoMxDrXPo9DHvhie_P2e2z5vOPXT0tOx92c_ZE5Mmmp9To_hCDohwUQdvuD3tsegj4lXQWR Im0sj8aWEe6HgkwRY7kM.E9jqQ1u_NROQwWVdzaaAaO5Azph0M7wfRtfhyYFUv51CKHr704j3DVz vaWodnlzkGwtLd8s0jxkk6BuHV4WZXunPm94qZdCmrbGtuLCQzSqUVqxEv9SxfS3hi.s8bnlv7Yw Tl3Zn3zzzw2.1xVsQgUFTZjH7ghNIvS.rsmem19jL07pWULdA91Fa6QbfueVWgWTCwLo3v_1qc3K exk92B64xTdm.3JF5nsV4zWnrn9_Hnf0KkrpBXilcdTU9zUA3TaGTqKfpvg0sFUWfHZ5rEyjLhYz 4OVmgd9sEFMcANXl0QChMqbNYqFBrGr0IXX3eXu_LvLFYUoX.dscOaV0CAevlkES9C4yi_Y3U2cG 2kmXiH0oK_sGMEaZ1I3h9PgRJonBowZBVgyp6WjFXyl6hzUSM5Kcsvx9oaGD4Fvwkkn4qsnUi9Av 3he8o X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic313.consmr.mail.gq1.yahoo.com with HTTP; Thu, 23 Feb 2023 20:23:20 +0000 Received: by hermes--production-bf1-57c96c66f6-kqcsw (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 5cee844f486519a9cd19cf1ab6236030; Thu, 23 Feb 2023 20:23:14 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: FYI: Why META_MODE rebuilds so much for building again after installworld (no source changes) [code level bug evidence] From: Mark Millard In-Reply-To: <266ED18F-9249-46BB-BF96-1D4C5B46FCFC@yahoo.com> Date: Thu, 23 Feb 2023 12:23:02 -0800 Cc: Bryan Drewery , Current FreeBSD , Peter Content-Transfer-Encoding: quoted-printable Message-Id: References: <3345EBA5-A09C-4E3F-B94D-39F57F56BDBB@yahoo.com> <73088.1611797582@kaos.jnpr.net> <10819.1677108389@kaos.jnpr.net> <76FA98EF-6184-4D7E-A01F-0EE8117D0D10@yahoo.com> <29887.1677115125@kaos.jnpr.net> <27790339-240F-4C97-97C7-38AFD8DE03D5@yahoo.com> <72419.1677133429@kaos.jnpr.net> <266ED18F-9249-46BB-BF96-1D4C5B46FCFC@yahoo.com> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-3.46 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.96)[-0.960]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.83:from]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-current@FreeBSD.org] X-Rspamd-Queue-Id: 4PN4Gf3qjMz4LnB X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On Feb 23, 2023, at 11:53, Mark Millard wrote: > cached_realpath only reports its "cached_realpath:" notice > (not the purging one) when it does not find the value via > HashTable_FindValue and so does a HashTable_Set : >=20 > const char * > cached_realpath(const char *pathname, char *resolved) > { > const char *rp; >=20 > if (pathname =3D=3D NULL || pathname[0] =3D=3D '\0') > return NULL; >=20 > rp =3D HashTable_FindValue(&cached_realpaths, pathname); > if (rp !=3D NULL) { > /* a hit */ > strncpy(resolved, rp, MAXPATHLEN); > resolved[MAXPATHLEN - 1] =3D '\0'; > return resolved; > } >=20 > rp =3D realpath(pathname, resolved); > if (rp !=3D NULL) { > HashTable_Set(&cached_realpaths, pathname, = bmake_strdup(rp)); > DEBUG2(DIR, "cached_realpath: %s -> %s\n", pathname, = rp); > return resolved; > } >=20 > /* should we negative-cache? */ > return NULL; > } >=20 > cached_realpaths is global: >=20 > static HashTable cached_realpaths; >=20 > So with -ddM why do I see lots of "cached_realpath:" > notices for the same path? For example: >=20 > # grep "tmp/legacy/usr/sbin/ln\>" = /usr/obj/BUILDs/main-amd64-nodbg-clang/sys-typescripts/typescript-make-amd= 64-nodbg-clang-amd64-host-2023-02-23:10:20:26 | more > cached_realpath: = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln -> = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /bin/ln > cached_realpath: = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln -> = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /bin/ln > Caching 02:49:37 Feb 23, 2023 for = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln > = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/usr.bin/aw= k/awkgram.tab.h.meta: 22: file = '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac= y/usr/sbin/ln' is newer than the target... > cached_realpath: = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln -> = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /bin/ln > Caching 02:49:37 Feb 23, 2023 for = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln > cached_realpath: = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln -> = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /bin/ln > Caching 02:49:37 Feb 23, 2023 for = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln > cached_realpath: = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln -> = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /bin/ln > Caching 02:49:37 Feb 23, 2023 for = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln > cached_realpath: = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln -> = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /bin/ln > Caching 02:49:37 Feb 23, 2023 for = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln > cached_realpath: = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln -> = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /bin/ln > Caching 02:49:37 Feb 23, 2023 for = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy= /usr/sbin/ln > = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/usr.bin/aw= k/awkgram.tab.h.meta: 22: file = '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac= y/usr/sbin/ln' is newer than the target... > . . . >=20 > A possible cause is something I ran into while looking around: >=20 > /* A read-only range of a character array, NOT null-terminated. */ > typedef struct Substring { > const char *start; > const char *end; > } Substring; > . . . > MAKE_STATIC Substring > Substring_Init(const char *start, const char *end) > { > Substring sub; >=20 > sub.start =3D start; > sub.end =3D end; > return sub; > } > . . . > /* Find the entry corresponding to the key, or return NULL. */ > HashEntry * > HashTable_FindEntry(HashTable *t, const char *key) > { > const char *keyEnd; > unsigned int h =3D Hash_String(key, &keyEnd); > return HashTable_Find(t, Substring_Init(key, keyEnd), h); > } > . . . > /* A read-only range of a character array, NOT null-terminated. */ > typedef struct Substring { > const char *start; > const char *end; > } Substring; > . . . > MAKE_STATIC Substring > Substring_Init(const char *start, const char *end) > { > Substring sub; >=20 > sub.start =3D start; > sub.end =3D end; > return sub; > } > . . . > /* Find the entry corresponding to the key, or return NULL. */ > HashEntry * > HashTable_FindEntry(HashTable *t, const char *key) > { > const char *keyEnd; > unsigned int h =3D Hash_String(key, &keyEnd); > return HashTable_Find(t, Substring_Init(key, keyEnd), h); > } > . . . > /* This hash function matches Gosling's Emacs and java.lang.String. */ > static unsigned int > Hash_String(const char *key, const char **out_keyEnd) > { > unsigned int h; > const char *p; >=20 > h =3D 0; > for (p =3D key; *p !=3D '\0'; p++) > h =3D 31 * h + (unsigned char)*p; >=20 > *out_keyEnd =3D p; > return h; > } >=20 > But after the loop: *p=3D=3D'\0' so *out_keyEnd=3D=3D'\0' > and the FindEntry Substring_Init(key, keyEnd) ends > up including the '\0' byte. >=20 > But note that the h in Hash_String did not include the > '\0' byte. Call this h value h_VALUE0 for later reference. > Then look at: >=20 > /* This hash function matches Gosling's Emacs and java.lang.String. */ > unsigned int > Hash_Substring(Substring key) > { > unsigned int h; > const char *p; >=20 > h =3D 0; > for (p =3D key.start; p !=3D key.end; p++) > h =3D 31 * h + (unsigned char)*p; > return h; > } >=20 > This h does include the '\0' byte so h=3D=3D(unsigned = int)(31*h_VALUE0). Dumb mistake on my part. Actually *(key.end) is never used., even if *(key.end) !=3D '\0' . > I expect the mismatched hash values explain the repeated > "cached_realpath:" notices for the same path: inserted > but never found. Still, the comments and code do not match and I've not checked all usage for assumptions about *(key.end) vs. '\0' . =3D=3D=3D Mark Millard marklmi at yahoo.com