Big physically contiguous mbuf clusters

[Rick Macklem]

Garrett Wollman wrote:
> In article <20140129231138$3db6 at grapevine.csail.mit.edu>,
> nparhar at gmail.com writes:
> 
> >I think this would be very useful.  For example, a zone_jumbo32
> >would
> >hit a sweet spot -- enough to fit 3 jumbo frames and some loose
> >change
> >for metadata.  I'd like to see us improve our allocators and VM
> >system
> >to work better with larger contiguous allocations, rather than
> >deprecating the larger zones.  It seems backwards to push towards
> >smaller allocation units when installed physical memory in a typical
> >system continues to rise.
> 
> In order to resist fragmentation, you need to be willing to dedicate
> some partition of physical memory to larger allocations.  That's fine
> for a special-purpose device like a switch, but is not so good for a
> general-purpose operating system.  But if you were willing to
> reserve,
> say, 1/64th of physical memory at boot time, make it all
> direct-mapped
> using superpages, and allocate it in fixed-power-of-two-sized chunks,
> you would probably get a performance win.  But the chunks *have* to
> be
> fixed-size, otherwise you are nearly guaranteed to get your arena
> checkerboarded.  I'd consider giving 2 GB on a 128-GB machine for
> that.
> 
> For NFS performance, you'd probably want to be able to take a whole
> chunk, read the desired data into it in a single VOP, then pass the
> whole thing to the socket layer wrapped in an mbuf.
> 
Yep, 1 64K (or 128K soon) mbuf would be nice for read, readdir, write.
(Assuming tcp_output knows how to split it up for net interfaces that
 can't handle TSO segments that large.)

I'm not sure why, but most use 65535 (max IP datagram size) as if_hw_tsomax.
(This guarantees the 64K NFS send gets split up. Doesn't TSO split it up
 into mtu sized segments? If so, I don't see why if_hw_tsomax would be
 a limit?) I'm not knowledgible w.r.t. TSO, so feel free to ignore or
ocrrect this.

rick

> -GAWollman
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to
> "freebsd-net-unsubscribe at freebsd.org"
>