FreeBSD hugepages

Hi there,

I have been steadily working on bringing Data Plane Development Kit 
(DPDK) on FreeBSD up to date with the Linux version. The most 
significant hurdle so far has been supporting concurrent DPDK processes, 
each with their own contiguous memory regions.

These contiguous regions are used by DPDK as a heap for allocating DMA 
buffers and other miscellaneous resources. Retrieving the underlying 
memory and mapping these regions is currently different on Linux and 
FreeBSD:

On Linux, hugepages are fetched from the kernel's pre-allocated hugepage 
pool and are mapped into virtual address space on DPDK initialization. 
Since the hugepages exist in a pool, multiple processes can reserve 
their own hugepages and operate concurrently.

On FreeBSD, DPDK uses an in-house contigmem kernel module that reserves 
a large contiguous region of memory on load. During DPDK initialization, 
the entire region is mapped into virtual address space. This leaves no 
memory for another independent DPDK process, so only one process can 
operate at a time.

I could modify the DPDK contigmem module to mimic Linux's hugepages, but 
I thought it would be better to integrate and upstream a hugepage-like 
interface directly in the FreeBSD kernel source. I am writing this email 
to see if anyone has any advice on the matter. I did not see any 
previous attempts at this in Phabriactor or the commit log, but it is 
possible that I missed it. I have read about transparent superpage 
promotion, but that seems like a different mechanism altogether.

At a quick glance, the implementation seems straightforward: read some 
loader tunables, allocate persistent hugepages at boot time, and create 
a pseudo filesystem that supports creating and mapping hugepages. I 
could be underestimating the magnitude of this task, but that is why I'm 
asking for thoughts and advice :)

For reference, here is Linux's documentation on hugepages:
https://docs.kernel.org/admin-guide/mm/hugetlbpage.html

Let me know if you have any thoughts,
Jake Freeland