kernel RPC over netlink(4) and general krpc cleanup

From: Gleb Smirnoff <glebius_at_freebsd.org>
Date: Mon, 06 Jan 2025 19:10:19 UTC
  Hi,

last year I tried to check-in a new implementation of unix(4) SOCK_STREAM and
SOCK_SEQPACKET in d80a97def9a1, but was forced to back it out due to several
kernel side abusers of a unix(4) socket.  The most difficult ones are the NFS
related RPC services, that act as RPC clients talking to an RPC servers in
userland.  Since it is impossible to fully emulate a userland process
connection to a unix(4) socket they need to work with the socket internal
structures bypassing all the normal KPIs and conventions.  Of course they
didn't tolerate the new implementation that totally eliminated intermediate
buffer on the sending side.

As I want to go forward with the new unix/stream and unix/seqpacket I need to
do something with the kernel RPC.  Today we got a new kind of socket -
netlink(4) that is designed specifically for kernel<->userland communication.
Although it is originally designed to provide kernel services to userland, we
can work it around to do the opposite.  The plan is that kernel modules that
are seeking a specific RPC service will multicast their requests on specific
netlink multicast groups and respective userland programs will reply on them.

Working on that idea I realized that the kernel RPC code (living in sys/rpc)
has quite a lot of dead code.  Some is disabled at compile time, some is
basically never called or never reached. Thus, in combination with two new
modules: kernel netlink client and libc/rpc netlink server, I am also going to
do some code deletion in sys/rpc.  Note that I don't want to refactor anything,
I just want to shave off pieces that are never used.  This cleanup makes it
much easier to understand what needs to be done to avoid abuse of unix(4)
socket and what doesn't.

My development branch is shared here:

https://github.com/glebius/FreeBSD/commits/gss-netlink/

ATM, it has converted to netlink(4) communication for gssd(8) and rpcbind(8).

Comments welcome!

-- 
Gleb Smirnoff