180,000 kevents - out of memory errors

[Terry Lambert]

Simon Lai wrote:
> I want to watch approx 200,000 sockets
> using kevent, but I get an ENOMEM error
> when adding kevent structures to the kqueue.
> The machine in question has 1.5GB RAM
> and can happily accept 270,000 TCP connections,
> but I need kevent to manage that many sockets,
> select() being a little too slow.
> 
> I want to add one kevent struct to the queue
> for each connection.
> 
> Currently I can only queue around 80,000 - 90,000
> kevents, with this config -
> 
>    maxfiles=250000
>    nmbclusters=35000
>    nmbufs=210000
> 
> with this config I can make 180,000 connections and
> queue 180,000 kevents -
> 
>    maxfiles=200000
>    nmbclusters=40000
>    nmbufs=240000
> 
> what should I do to improve the number of kevents
> I can queue?

1)	Add 2.5G more RAM.

2)	Reconfigure the KVA space to use 1G for UVA and 3G for
	KVA, instead of 2G/2G, which is the default, so you
	won't run out of page mappings.

3)	Disable IPSEC.

4)	Separate the allocation of tcpcb's and udpcb's; there
	is no reason to support 200,000 for both, when you can
	support 200,000 for one, and 1,000 or less for the other,
	and save all that memory.

-- Terry