Frequent hickups on the networking layer

[Adrian Chadd]

I've spoken to more than one company about this stuff and their
answers are all the same:

"we ignore the freebsd allocator, allocate a very large chunk of
memory at boot, tell the VM it plainly just doesn't exist, and abuse
it via the direct map."

That gets around a lot of things, including the "oh how can we get 9k
allocations if we can't find contiguous memory/KVA/either" problem -
you just treat it as an array of 9k pages (or I'm guessing much larger
- as you said, like ~ 64k), and allocate that way. That way there's no
fragmentation to worry about - everything's just using a custom slab
allocator for these large allocation sizes.

It's kind of tempting to suggest freebsd support such a thing, as I
can see increasing requirements for specialised applications that want
this. One of the things that makes netmap so nice is it 100% avoids
the allocators in the hot path - it grabs a big chunk of memory and
allocates slots out of that via a bitmap and index values.




-adrian