Re: FYI: Why META_MODE rebuilds so much for building again after installworld (no source changes)

[I'm back at this again after some time on other subjects.]

On Feb 23, 2023, at 17:42, Simon J. Gerraty <sjg@juniper.net> wrote:

> Mark Millard <marklmi@yahoo.com> wrote:
> 
>> Simplifying context . . .
>> . . .
>>> As I mentioned previously, there is no variablity of OBJTOP within the
>>> context of a single make instance - at least not once it starts running
>>> targets.
>>> 
>>>> . . .
>>> 
>>> .MAKE.META.IGNORE_PATHS += ${OBJTOP}/tmp/legacy/usr
>> 
>> I'll use that definition line for the below.
> 
> Ok.
> 
>>> should result in nothing under ${OBJTOP}/tmp/legacy/usr causing a target
>>> to be out of date - just because it is newer.
>> 
>> I'll ignore there that that is skipping too much
>> and just show what happens for the 2nd buildkernel
>> of 2 in a row when I use that exact line for both
>> make runs.
>> 
>> First counts of the "is newer than" lines, counting
>> separate program names separately:
>> 
>> # cat /usr/obj/BUILDs/main-amd64-nodbg-clang/sys-typescripts/typescript-make-amd64-nodbg-clang-amd64-host-2023-02-23:16:15:18 | grep "is newer than the target" | sed -
>> e "s@^.*: file '@file '@" | sort | uniq -c | sort -rn | more
>> 2553 file '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy/usr/sbin/realpath' is newer than the target...
>> 1001 file '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy/usr/sbin/ln' is newer than the target...
>> 
> 
> It is of course critical to know what OBJDIR is at this point.
> also can you show me the line in the meta file that is matching.

So, using specific "is newer than" notice examples instead of
the above counts . . .

First some output log lines around a few sbin/realpath "is newer than"
related Building lines, with the .info lines in place now (I've
got both kmod.mk and kern.mk with the .info line, likely producing
redundant output but I did not know up front for sure):

make[4]: "/usr/main-src/sys/conf/kmod.mk" line 72: 
.CURDIR=/usr/main-src/sys/modules/aac
.OBJDIR=/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac
OBJTOP=/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src
.MAKE.META.IGNORE_PATHS=/bin/sh  /bin  /lib  /rescue  /sbin  /usr/bin  /usr/lib  /usr/sbin  /usr/share /usr/include /usr/local/etc/libmap.d /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.am
d64/sys/GENERIC-NODBG/modules/usr/main-src/tmp/legacy/usr
make[4]: "/usr/main-src/sys/conf/kern.mk" line 3:  .CURDIR=/usr/main-src/sys/modules/aac
.OBJDIR=/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac
OBJTOP=/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src
.MAKE.META.IGNORE_PATHS=/bin/sh  /bin  /lib  /rescue  /sbin  /usr/bin  /usr/lib  /usr/sbin  /usr/share /usr/include /usr/local/etc/libmap.d /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.am
d64/sys/GENERIC-NODBG/modules/usr/main-src/tmp/legacy/usr
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/machine.meta: 23: file '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy/usr/sbin/realpath' is newer than the target...
Building /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/machine
machine -> /usr/main-src/sys/amd64/include
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/x86.meta: 23: file '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy/usr/sbin/realpath' is newer than the target...
Building /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/x86
x86 -> /usr/main-src/sys/x86/include
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/i386.meta: 23: file '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy/usr/sbin/realpath' is newer than the target...
Building /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/i386
i386 -> /usr/main-src/sys/i386/include

Some lines just following the above are some sbin/ln "is newer than"
related Building lines:

Skipping meta for beforedepend: .PHONY
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/opt_scsi.h.meta: 12: file '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy/usr/sbin/ln' is newer than the target...
Building /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/opt_scsi.h
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/opt_cam.h.meta: 12: file '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy/usr/sbin/ln' is newer than the target...
Building /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/opt_cam.h
/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/opt_aac.h.meta: 12: file '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy/usr/sbin/ln' is newer than the target...
Building /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/opt_aac.h


For reference for a sbin/realpath "is newer than" example,
the sys/modules/aac/machine.meta :

# Meta data file /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/machine.meta
CMD @case machine in  machine)  path=/usr/main-src/sys/amd64/include ;;  *)  path=/usr/main-src/sys/machine/include ;;  esac ;  path=`realpath $path`;  echo machine "->" $path ;  ln -fns $path machine
CWD /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac
TARGET machine
-- command output --
machine -> /usr/main-src/sys/amd64/include

-- filemon acquired metadata --
# filemon version 5
# Target pid 84728
# Start 1677207916.433304
V 5
E 84729 /bin/sh
R 84729 /etc/libmap.conf
R 84729 /usr/local/etc/libmap.d
R 84729 /usr/local/etc/libmap.d/mesa.conf
R 84729 /var/run/ld-elf.so.hints
R 84729 /lib/libedit.so.8
R 84729 /lib/libc.so.7
R 84729 /lib/libtinfow.so.9
R 84729 /usr/share/locale/C.UTF-8/LC_CTYPE
F 84729 84730
E 84730 /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy/usr/sbin/realpath
R 84730 /etc/libmap.conf
R 84730 /usr/local/etc/libmap.d
R 84730 /usr/local/etc/libmap.d/mesa.conf
R 84730 /var/run/ld-elf.so.hints
R 84730 /lib/libc.so.7
X 84730 0 0
E 84729 /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy/usr/sbin/ln
R 84729 /etc/libmap.conf
R 84729 /usr/local/etc/libmap.d
R 84729 /usr/local/etc/libmap.d/mesa.conf
R 84729 /var/run/ld-elf.so.hints
R 84729 /lib/libc.so.7
D 84729 machine
L 84729 '/usr/main-src/sys/amd64/include' 'machine'
X 84729 0 0
# Stop 1677207916.435305
# Bye bye


For reference for a sbin/ln "is newer than" example,
the sys/modules/aac/opt_scsi.h.meta :

# Meta data file /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac/opt_scsi.h.meta
CMD ln -sf /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/opt_scsi.h opt_scsi.h
CWD /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/aac
TARGET opt_scsi.h
-- command output --

-- filemon acquired metadata --
# filemon version 5
# Target pid 84728
# Start 1677207916.439304
V 5
E 84735 /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy/usr/sbin/ln
R 84735 /etc/libmap.conf
R 84735 /usr/local/etc/libmap.d
R 84735 /usr/local/etc/libmap.d/mesa.conf
R 84735 /var/run/ld-elf.so.hints
R 84735 /lib/libc.so.7
D 84735 opt_scsi.h
L 84735 '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/opt_scsi.h' 'opt_scsi.h'
X 84735 0 0
# Stop 1677207916.440304
# Bye bye


> If you add the .info line I suggested in kern.mk or kmod.mk
> you should get some useful info.

Added to both for now. The material above has the output
from them from just before a "is newer than" notice.

>> Thousands of rebuilt things based on:
>> 
>> . . ./tmp/legacy/usr/sbin/realpath
>> . . ./tmp/legacy/usr/sbin/ln
>> 
>> It appears that buildkernel does not use an OBJTOP definition
>> that references:
> 
> It is possible it does not have a "normal" value for it.
> Kernel builds start in the objdir so .CURDIR is actually under
> OBJTOP, so unless the makefiles have appropriate logic for that case,
> and depending on how OBJTOP is derrived, you may have a bogus value.
> 
> The fix in that case would be the makefile setting OBJTOP.
> 
>> 
>> /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64
>> 
>> in my context.
>> 
>> For reference of Build lines paired with a few of those "is newer
>> than" lines:
>> 
>> 
>> I've still no clue of a notation that avoids this for
>> my choice to use personal MAKEOBJDIRPREFIX paths:
> 
> That should work, looking at share/mk/src.sys.obj.mk
> though looks like you might get
> 
> OBJROOT?=       ${_default_makeobjdirprefix}${SRCTOP}/
> 
> which may not be valid for a kernel build.
> At least ${.OBJDIR} will not have that ${OBJROOT} as a prefix
> and some of the settings for OBJTOP look just wrong to me:
> 
> OBJTOP:=        ${OBJROOT:H}
> 
> is the opposite of what the relationship b/w OBJROOT and OBJTOP are in
> DIRDEPS_BUILD but it looks like
> 
> OBJTOP:=        ${OBJROOT}${MACHINE}.${MACHINE_ARCH}
> 
> is more likely to be used, but given the above default for OBJROOT
> that is unlikely to work for a kernel build.
> 
> Looks like share/mk/src.sys.obj.mk grok's SB (the setup I/we use) so you
> can take control by setting SB and SB_OBJROOT to anything you like and
> it should be used for OBJROOT and presumably hence OBJTOP even for a
> kernel build.  Though I normally use MAKEOBJDIR (similar to the way it
> is set in _default_makeobjdir), I don't know how well that works with
> legacy targets though - there's a lot of baked in assumptions about
> using MAKEOBJDIRPREFIX.
> 
> Again that .info line I gave you would provide some useful clues.
> 

===
Mark Millard
marklmi at yahoo.com