kld support

[Peter Wemm]

On Wednesday 11 February 2004 01:01 am, Petri Helenius wrote:
> Since the kld support for amd64 is pending, is there a way to compile
> netgraph modules statically into the kernel to get for example
> ng_ether support?
>
> I would also be happy to try out early code if such thing exists. (to
> enable KLD)  For the time being, I have a spare opteron box.

I have bad news and some good news on this front.  I thought we might be 
able to get away with a quick hack to binutils to get it to work.  
Indeed, I've been able to produce .ko files that can be loaded and 
almost even work.  But thats the problem... almost.

Check out these lines in sys/conf/NOTES:
options         NETGRAPH                #netgraph(4) system
options         NETGRAPH_ASYNC
options         NETGRAPH_BPF
options         NETGRAPH_BRIDGE
options         NETGRAPH_CISCO
options         NETGRAPH_ECHO
options         NETGRAPH_ETHER
options         NETGRAPH_FRAME_RELAY
options         NETGRAPH_GIF
.....[lots more]....
Thats how you chose what you want to compile in.

Anyway, the problem with binutils is subtle but massive.  There is a 
sanity check that prevents you producing shared libs without -fpic 
mode.  Because *everybody* knows that ld.so cannot guarantee that it 
can load the library within the first 2GB of address space on linux, so 
binutils enforces this policy.  The problem though is that all the wold 
is not linux.  And there are legitimate reasons for doing this.  One of 
which is our kernel modules where we can provide those guarantees.

Anyway, removing the sanity check lets you produce them.  BUT... because 
those code paths have never been tested on linux, binutils neglects to 
support the rest of the relocation types that would be needed, and none 
of the linux folks have noticed.  Binutils is a freaking nightmare to 
try and work in, and it has me totally beaten on this one.

So, I thought..  since I can do something about the in-kernel linker, 
perhaps I can add PIC mode support?  The problem with that though is 
that gcc says "-fPIC mode not implemented in kernel mode" and errors 
out.  There's no way in hell that I'm going to try and implement that 
in gcc either, so plan B is a bust.

The good news though is that Plan C is possible.  Plan C is to stop 
using -shared for kernel modules and use a simple .o file, like linux 
does.  I think I can write code for the kld subsystem without too much 
pain.  And because linux does the same thing already, we dont have to 
worry about treading on landmines in binutils/gcc, because that path 
should be well trodden already.

We already produce these files in kmod.mk.  If you have a look at the 
obj dirs, we have:

ld  -d -warn-common -r -d -o if_sl.kld if_sl.o slcompress.o
touch /home/src/sys/modules/if_sl/obj/export_syms
awk -f /home/src/sys/modules/if_sl/../../conf/kmod_syms.awk 
if_sl.kld  /home/src/sys/modules/if_sl/obj/export_syms |  xargs -J% 
objcopy % if_sl.kld
ld -Bshareable  -d -warn-common -o if_sl.ko if_sl.kld
ld: if_sl.kld: relocation R_X86_64_32S can not be used when making a 
shared object; recompile with -fPIC
.... ls obj/
-rw-r--r--  1 root  src  19608 Feb 11 12:32 if_sl.kld

That file was the immediate step prior to conversion to a .ko file, and 
thats what we can load.

On another machine with the hacks to produce a .ko anyway, we have:
peter at hammer[12:37pm]/home/obj/home/src/sys/modules/if_sl-105> ls -l 
if_sl.*
-rw-r--r--  1 root  wheel  19608 Feb  6 14:46 if_sl.kld
-rwxr-xr-x  1 root  wheel  23179 Feb  6 14:46 if_sl.ko*
-rw-r--r--  1 root  wheel  15760 Feb  6 14:46 if_sl.o
-rw-r--r--  1 root  wheel  24668 Feb  6 14:46 if_sl.s

Also of note is that the .kld file is smaller than the .ko file.

So yes, there is light at the end of the tunnel.  I think its time to 
give up on binutils/gcc hacking and take the path of least resistence 
and write some code that we control.  The original reasons for having 
-shared kernel object files are mostly obsolete these days.  We dont 
use the DT_NEEDED tags for dependency tracking anymore, for example.
I think we can actually get a superior system as a result.
-- 
Peter Wemm - peter at wemm.org; peter at FreeBSD.org; peter at yahoo-inc.com
"All of this is for nothing if we don't go to the stars" - JMS/B5