Re: FYI: Why META_MODE rebuilds so much for building again after installworld (no source changes)

Mark Millard <marklmi@yahoo.com> wrote:
> >> Also, OBJTOP is not constant over all the parts of
> >> buildworld buildkernel . Having the late-substitution
> >> form of notation ${OBJTOP} might not be appropriate
> >> for the content of .MAKE.META.IGNORE_PATHS .
> >
> > Fwiw .MAKE.META.IGNORE_PATHS is evaluated when meta_init() is
> > called after all the makefiles have been read - and had a
> > chance to influence .MAKE.MODE, so I'm not sure how varaiablity of
> > OBJTOP would matter?
> 
> To my knowledge, there is no place to find documentation
> about when/how-often the .MAKE.META.IGNORE_PATHS original
> text is reevaluated other than the above statement or
> source code inspection. (Others have a similar status.)

True.  That should be fixed.
.MAKE.META.IGNORE_PATHS is only evaluated once by make
after all makefiles have been read.  It is use to populate a list.

By contrast .MAKE.META.IGNORE_PATTERNS and .MAKE.META.IGNORE_FILTER
are evaluated for every line of filemon output - but again all that
happens after all makefiles have been read.

> -dV -V.MAKE.META.IGNORE_PATHS use does list ${__MAKE_SHELL}
> but lists /bin/sh without the -dV . (So -V use does not
> give a direct clue at what you report.)

One of the local changes to bmake in FreeBSD is that 
without -dV -V shows the fully resolved value.

> I got past the issue using := before reading the above.
> (I'm also using MAKEOBJDIR instead of OBJTOP currently.)

Per my last response, I'd be pretty sure MAKEOBJDIR is incorrect.
> 
> >>> .MAKE.META.IGNORE_PATHS+= ${OBJTOP}/tmp/
> >>
> >> (Ignoring the variability of OBJTOP issue . . .)
> >>
> >> I do not expect that would work: ignoring things
> >> it likely should not.
> >
> > Sure, but it may be useful as an experiment to ensure things are
> > behaving as expected.
> 
> As a test:
> 
> .if ${.MAKE.LEVEL} == 0
> .MAKE.META.IGNORE_PATHS+= ${MAKEOBJDIR:tA}/tmp/
> .MAKE.META.IGNORE_PATHS:= ${.MAKE.META.IGNORE_PATHS}
> .endif

Lose the .if ${.MAKE.LEVEL} == 0
it is almost certainly keeping things from working as expected.

> I still get things like:
> 
> /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/zlib/x86.meta: 23: file '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy/usr/sbin/realpath' is newer than the target...
> Building /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/zlib/x86

Because that will not be level 0 and so .MAKE.META.IGNORE_PATHS is not
set.

> 
> and:
> 
> /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/xl/opt_platform.h.meta: 12: file '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legacy/usr/sbin/ln' is newer than the target...
> Building /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERIC-NODBG/modules/usr/main-src/sys/modules/xl/opt_platform.h
> 
> for both of a pair of back-to-back runs of buildworld buildkernel.
> 
> FYI: The file system is zfs with mounts that look
> like:
> 
> zoptb                           /zoptb
> zoptb/BUILDs                    /usr/obj/BUILDs
> . . .
> zoptb/BUILDs/main-amd64-nodbg-clang  /usr/obj/BUILDs/main-amd64-nodbg-clang
> . . .
> zoptb/ROOT/main-amd64           /
> . . .
> zoptb/tmp                       /tmp
> . . .
> 
> # bectl list
> BE             Active Mountpoint Space Created
> 13S-amd64      -      -          4.97G 2021-08-20 16:57
> 13_0R-amd64    -      -          4.30G 2021-08-20 16:56
> 13_1R-amd64    -      -          4.12G 2022-03-10 12:38
> main-amd64     NR     /          7.42G 2023-02-19 15:37
> old-main-amd64 -      -          2.25G 2023-02-09 19:07
> 
> (I use zfs in order to use bectl on a couple of
> systems, not for redundancy.)
> 
> 
> >> Also, I'd rather grow a smaller set of ignores
> >> gradually to make it easier to detect if an
> >> addition starts causing a problem and can be
> >> backed out. Starting with everything ignored
> >> would make things much harder to figure out
> >> when ignoring creates a problem.
> >
> > Yes.
> >
> >>
> >>> You might need ${OBJTOP:tA}/tmp/
> >>> or both.
> >
> > I found it necessary in the unit tests to add :tA to both TMPDIR
> > and .OBJDIR to get sane result on one test platform.
> >
> >>>> It is using paths that match the -dM output lines ( sbin
> >>>> use despite sbin -> ../bin being a symbolic link).
> >
> > use :tA if you want to ensure consistent results.
> 
> So, for each:
> 
> .MAKE.META.IGNORE_PATHS+= ${MAKEOBJDIR}/tmp/legacy/usr/sbin/${ignore_legacy_tool}
> 
> I need to form an overall :tA on the path? Something
> like:
> 
> .if ${.MAKE.LEVEL} == 0

I think you need to first get rid of that level 0 check before
worrying about anything else.

> .for ignore_legacy_tool in awk cap_mkdb cat cp crunchgen crunchide dd egrep env file2c gencat grep gzip jot lex lb ln m4 mkcsmapper mktemp mv patch realpath rm sed sh touch truncate uudecode uuencode
> xargs
> IGNORELEGACY_${ignore_legacy_tool}= ${MAKEOBJDIR}/tmp/legacy/usr/sbin/${ignore_legacy_tool}
> .MAKE.META.IGNORE_PATHS+= ${IGNORELEGACY_${ignore_legacy_tool}:tA}
> .endfor
> .for ignore_other_tool in ctfconvert objcopy nm
> IGNOREOTHER_${ignore_other_tool}= ${MAKEOBJDIR}/tmp/usr/bin/${ignore_other_tool}
> .MAKE.META.IGNORE_PATHS+= ${IGNOREOTHER_${ignore_other_tool}:tA}
> .endfor
> .MAKE.META.IGNORE_PATHS:= ${.MAKE.META.IGNORE_PATHS}
> .endif
> 
> Such seems to make no difference to the text reported via
> -dV -V.MAKE.META.IGNORE_PATHS in my context.

Yes, right now I think your main problem is only setting
.MAKE.META.IGNORE_PATHS at level 0