pf & NAT issue

[Bakul Shah]

On Fri, 20 Jan 2017 13:12:07 PST =?UTF-8?Q?Ermal_Lu=C3=A7i?= <eri at freebsd.org> wrote:
> --001a1148cecc40685805468d1ad2
> Content-Type: text/plain; charset=UTF-8
> 
> On Fri, Jan 20, 2017 at 12:59 PM, Bakul Shah <bakul at bitblocks.com> wrote:
> 
> > On Fri, 20 Jan 2017 21:43:33 +0100 "Kristof Provost" <kp at FreeBSD.org>
> > wrote:
> > > On 20 Jan 2017, at 21:31, Bakul Shah wrote:
> > > >> 11:56:28.168693 IP 192.168.125.7.65042 > 149.20.1.200.21: Flags [P.],
> > > >> seq 1:10, ack 55, win 1026, options [nop,nop,TS val 198426 ecr
> > > >> 1468113725], length 9
> > > > < 11:56:28.168712 IP 173.228.5.8.52015 > 149.20.1.200.21: Flags [P.],
> > > > seq 3080825147:3080825156, ack 3912707414, win 1026, options
> > > > [nop,nop,TS val 198426 ecr 1468113725], length 9
> > > >
> > > >     Right here we see the problem. NAT mapping for the
> > > >     port changed from 63716 to 52015.
> > > >
> > > Changing source ports is an entirely normal NAT behaviour.
> > >
> > > The best explanation is this: imagine that you have two clients A and B,
> > > both connect to X on port 80 via the NAT gateway G.
> > > Both use port 1000 as their source port.
> > > A connects, and the gateway maps A:1000 -> X:80 to G:1000 -> X:80.
> > > B connects, and now the gateway has to map B:1000 -> X:80 onto G:1000 ->
> > > X:80, but then it wouldn't be able to tell the two connections apart.
> > > That't can remap it onto G:1001 -> X:80 instead.
> >
> > It is the same connection!  As a tcp connection is identified
> > by <src ip, src port, dst ip, dst port>, If the port number
> > changes on the same connection, the remote side would see this
> > as a separate connection.
> >
> 
> Most probably your timeouts are aggressive on states garbage collection.
> Give a look to those state limit teardown it might improve things.

$ pfctl -s timeout
tcp.first                   120s
tcp.opening                  30s
tcp.established           86400s
tcp.closing                 900s
tcp.finwait                  45s
tcp.closed                   90s
tcp.tsdiff                   30s
udp.first                    60s
udp.single                   30s
udp.multiple                 60s
icmp.first                   20s
icmp.error                   10s
other.first                  60s
other.single                 30s
other.multiple               60s
frag                         30s
interval                     10s
adaptive.start             6000 states
adaptive.end              12000 states
src.track                    30s

local port num changed after 23 seconds.  All the tcp.*
timeouts seem ok. IIRC internal is used for IP frags.

Thanks
Bakul