More fine-grained NUMA knobs

[Konstantin Belousov]

On Tue, Mar 22, 2016 at 05:13:05PM -0700, John Baldwin wrote:
> Some of the I/O device affinity stuff such as bus_get_domain() (and the
> bus_get_cpus() I have in review in D5519) are useful on their own right even
> if the VM system is not doing NUMA-aware allocations.  I think it would be
> useful to be able to enable these two "prongs" of NUMA awareness
> independently.  To that end, I have a little strawman patch that adds two
> new kernel options: VM_NUMA_ALLOC and DEVICE_NUMA.  I actually think it is
> probably worth enabling DEVICE_NUMA by default on x86 (and bumping the
> default MAXMEMDOM to, say, 8 (quad-socket haswell)).
> 
> You can see the simple patch at:
> 
> https://github.com/bsdjhb/freebsd/compare/master...bsdjhb:numa_opts
> 
> Thoughts?

I do not like that you check both defined(xxx_NUMA) and MAXMEMDOM > 1.
IMO the > 1 part should be dropped.  I do not see why would it be useful
even as optimization.

Otherwise, this looks good.