[Development report #6] Improve the kinst DTrace provider

From: Christos Margiolis <christos_at_freebsd.org>
Date: Tue, 28 Mar 2023 12:52:27 UTC
The past few days I've been working on various bug fixes both in kinst
and in libdtrace. Inline function tracing is almost done [1].

libdtrace now parses all loaded kernel modules, instead of just
`kernel`. This makes it compatible with kinst, which also searches all
loaded modules.

After some testing I noticed that it is possible to have both a
non-inline and an inline definition of the same function in a kernel
module. If libdtrace finds such a case, it creates an additional FBT
probe for the non-inline definition:

	# dtrace -dn 'kinst::cam_strvis_flag:entry'
	kinst::cam_strvis:25,
	kinst::cam_strvis_flag:0,
	fbt::cam_strvis_flag:entry
	{
	}

	dtrace: description 'kinst::cam_strvis_flag:entry' matched 3
	probes

For inline tracing, I implemented the algorithm described here [2]
(markj@ has also implemented it in lib/libproc/proc_sym.c) to make sure
that both the modules' ELF and debug files are up to date (i.e the
module has been built with `DEBUG_FLAGS=-g`), otherwise we might run
into version mismatches between functions. If such a mismatch is found,
libdtrace prints a warning and skips that module.

I wrote a few Kyua tests for kinst and made use of
sys/dev/dtrace/dtrace_test.c (see inline tracing PR).

In my previous email I mentioned that I modified kinst to search for
`push %rbp` anywhere in a function, and skip the function if no `push
%rbp` is found. Since this affects safe-to-trace functions that do
not `push %rbp`, I'm working on an experimental change to exclude only
exception handlers, and not search for `push %rbp` at all [3]. However,
I'm still not sure this is 100% fail-proof, and I will need to do some
more testing to make sure there are no accidental crashes.

[1] https://reviews.freebsd.org/D38825
[2] https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html
[3] https://reviews.freebsd.org/D39229