maximum number of outgoing connections

[Igor Sysoev]

On Mon, Aug 20, 2007 at 09:53:55AM -0700, John-Mark Gurney wrote:

> Igor Sysoev wrote this message on Mon, Aug 20, 2007 at 19:11 +0400:
> > It seems that FreeBSD can not make more than
> > 
> > net.inet.ip.portrange.last - net.inet.ip.portrange.first
> > 
> > simultaneous outgoing connections, i.e., no more than about 64k.
> > 
> > If I made ~64000 connections 127.0.0.1:XXXX > 127.0.0.1:80, then
> > connect() to an external address returns EADDRNOTAVAIL.
> 
> Isn't this more of a limitation of TCP/IP than FreeBSD?  because you
> need to treat the srcip/srcport/dstip/dstport as a unique value, and
> in your test, you are only changing one of the four...  Have you tried
> running a second we server on port 8080, and see if you can connect
> another ~64000 connections to that port too?

No, TCP/IP limitation is for XXXX in 127.0.0.1:XXXX <> 127.0.0.1:80,
but FreeBSD limits all outgoing connections to the port range, i.e.

    local part      remote part
  127.0.0.1:5000 <> 127.0.0.1:80
192.168.1.1:5000 <> 10.0.0.1:25

can not exist simultaneously, if both connections were started from
local host.

I can not write a simple test-case program, but I can offer simple setup:

cd /usr/ports/www/nginx && make install

create simple nginx.conf:

------------
events {
    worker_connections  20000;
}

http {
    server {
        listen        8080;
        server_name   test;

        location = /loop {
            proxy_pass  http://127.0.0.1:8080;

            error_page  502 = /yahoo;
        }

        location = /yahoo {
            proxy_pass  http://www.yahoo.com;
        }
    }
}
------------

set

sysctl net.inet.ip.portrange.randomized=0
sysctl net.inet.ip.portrange.first=1024
sysctl net.inet.ip.portrange.last=5000

to see the case with default small number of files, sockets, etc.

and run as root:

/usr/local/sbin/nginx -c ./nginx.conf

then ask http://host:8080/loop in browser. nginx will cycle to itslef, then
after first error

2007/08/20 22:05:16 [crit] 29669#0: *94165 connect() to 127.0.0.1:8080 failed (49: Can't assign requested address) while connecting to upstream, client: 127.0.0.1, server: test, URL: "/loop", upstream: "http://127.0.0.1:8080/loop", host: "127.0.0.1:8080"

you will see the second error:

2007/08/20 22:05:16 [crit] 29669#0: *94165 connect() to 87.248.113.14:80 failed (49: Can't assign requested address) while connecting to upstream, client: 127.0.0.1, server: test, URL: "/loop", upstream: "http://87.248.113.14:80/loop", host: "127.0.0.1:8080"

If you think it may be nginx fault, run this under ktrace/truss and see
syscalls.


-- 
Igor Sysoev
http://sysoev.ru/en/