Kernel tuning for large maxsockets

[Scot Loach]

Bosko:

The problem I have is that each of the four pcb types has its own zone that
is preallocated to hold maxsockets pcbs.  This is a waste of kva that could
be better used.  Since I'm not using divert sockets or raw sockets, I would
rather cap these zones to a small constant number, and increase the others
as much as possible.

I should have a patch for this next week sometime.

-----Original Message-----
From: Bosko Milekic [mailto:bmilekic at technokratis.com]
Sent: Wednesday, July 16, 2003 5:52 AM
To: Scot Loach
Cc: 'freebsd-net at freebsd.org'
Subject: Re: Kernel tuning for large maxsockets



On Tue, Jul 15, 2003 at 05:22:55PM -0400, Scot Loach wrote:
> Currently, whenever maxsockets is increased, this causes kernel memory to
be
> preallocated for each type of pcb (tcp, udp, raw, divert).  The number of
> pcbs preallocated for each of these is always the same as maxsockets.
> 
> This is probably a waste of memory for raw sockets and divert sockets,
since
> they would not normally be used in large numbers.   A large server could
> save kvm by reducing the number of divert and raw pcbs preallocated.  For
> example, on a machine configured for 200,000 maxsockets we would save 75MB
> of kvm out of a total of 262MB that would have been preallocated.  This
kvm
> savings can be used to increase maxsockets even more.

  What version of FreeBSD are you talking about here?  In -current, the
  pcbs come off of zones which are capped at a maximum w.r.t.
  maxsockets.  The kva space comes out of kmem_map and the objects are
  kept cached in their respective zones.  One thing to note is that the
  zones are setup with UMA_ZONE_NOFREE, so the pages wired down for pcb
  use are probably never unwired.

  I don't know why, exactly, UMA_ZONE_NOFREE is required for all of the
  pcb zones in the netinet code.

> Is there any reason I should not modify the kernel code to only let a
small,
> fixed number of raw and divert pcbs be preallocated instead of having them
> scale with maxsockets?
> 
> Next, does this seem like a generally useful thing that could be rolled
back
> into the source tree?  I could make this a kernel option or a tunable
sysctl
> variable.
> 
> thanks
> 
> Scot Loach

-- 
Bosko Milekic  *  bmilekic at technokratis.com  *  bmilekic at FreeBSD.org
TECHNOkRATIS Consulting Services  *  http://www.technokratis.com/