Limits on jumbo mbuf cluster allocation

[Garrett Wollman]

<<On Fri, 08 Mar 2013 08:54:14 +0100, Andre Oppermann <andre at freebsd.org> said:

> [stuff I wrote deleted]
> You have an amd64 kernel running HEAD or 9.x?

Yes, these are 9.1 with some patches to reduce mutex contention on the
NFS server's replay "cache".

> Jumbo pages come directly from the kernel_map which on amd64 is 512GB.
> So KVA shouldn't be a problem.  Your problem indeed appears to come
> physical memory fragmentation in pmap.

I hadn't realized that they were physically contiguous, but that makes
perfect sense.

> pages.  Also since you're doing NFS serving almost all memory will be
> in use for file caching.

I actually had the ZFS ARC tuned down to 64 GB (out of 96 GB physmem)
when I experienced this, but there are plenty of data structures in
the kernel that aren't subject to this limit and I could easily
imagine them checkerboarding physical memory to the point where no
contiguous three-page allocations were possible.

-GAWollman