[Bug 265019] pmap_growkernel wiring excess memory on module load

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=265019

            Bug ID: 265019
           Summary: pmap_growkernel wiring excess memory on module load
           Product: Base System
           Version: CURRENT
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: ashafer@badland.io

When loading a large kernel module (in this case nvidia.ko) a massive amount of
memory will be fired. This ends up wiring 4.2Gb out of 8Gb on the system, which
significantly hurts usability.

There was a lot of discussion on discord about this, I'll try to include it
here:
https://discord.com/channels/727023752348434432/757305573866733680/992940224063746058

"Maybe it is the addr param itself after all. I'm seeing some weirdness in the
min address used for kernel_vm_end that I haven't explained yet. KERNBASE is
0xffffffff80000000, but I see vm_map_find using 0xfffffe0000000000 as the min
address to start searching for open space at. So kernel_vm_end appears to start
at fffffe00b1600000 when loading the kernel (the kernel log drops some lines
there so this is the first value I see for it), and it's raised to find more
space. That seems to be the general "neighborhood" of addresses passed to
pmap_growkernel . Then at some point when I call kldload it passes some offset
from KERNBASE instead of from that "neighborhood", and since KERNBASE and
0xfffffe0000000000 are so far away it allocates a million pages.
Sorry the logic there is a little fuzzy, but tldr is it feels like there are
two different ideas about where the kernel VM address starts and kldload
triggers us using the wrong one."

What we think is happening is that normally vm_map_find is called starting from
VM_MIN_KERNEL_ADDRESS, but link_elf_load_file starts searching at KERNBASE.
This ends up wiring massive amount of memory to split the difference. Normally
this would be fine since we reserve a bunch of pages after KERNBASE for kernel
modules, but if the kernel module is too large and overflows this region, this
error might occur.

This PR tracks improving the logic of pmap_growkernel to properly track growing
in multiple ways. We might want to have two kernel_vm_ends, and decide the
starting/ending points of growing the kernel based on the address given to
pmap_growkernel and if it's above KERNBASE

-- 
You are receiving this mail because:
You are the assignee for the bug.