Mysterious packets with stateful ipfw+nat

[James Halstead]

Ok, the "obvious" part that I think I was missing while it was late, was 
that these must be keep-alive packets generated by the firewall as the 
dynamic rules are about to expire. That being the case however, 
shouldn't these keep-alive packets take the same action as the original 
rule (skipto 1000 and be diverted through NAT for processing)?

James Halstead wrote:
> Ok, this has been driving me nuts for a while. I recently noticed that 
> my 5.4-RELEASE firewall was having a problem with packet "leakage". I am 
> seeing the occasional packet on the outside interface with an internal 
> src ip. I put a hub between my firewall and cable modem and verified 
> that the packets are indeed on the wire. Now I am in the process of 
> setting up a new 6.1-RELEASE box and the same issue was happening on my 
> test network.
> 
> So far I don't get it. I must be missing something obvious. At least 
> everything still works in general.
> 
> The test setup is a clean install of 6.1-RELEASE, using GENERIC with the 
> ipfw.ko and ipdivert.ko modules loaded. After searching around I was 
> basing the configuration off of:
> 
> http://lists.freebsd.org/mailman/htdig/freebsd-ipfw/2004-June/001182.html
> 
> The test box has two Ethernet interfaces, renamed to be isp0 and net0. 
> isp0 is using DHCP, and receives the address 10.42.0.220/24. net0 is 
> running a DHCP server, and sits on 192.168.1.1/24. There is one single 
> piece of hardware on net0 which is always assigned 192.168.1.230. The 
> gateway to the actual Internet sits on 10.42.0.254. A pretty simple setup.
> 
> The internal machine is just constantly connecting to an external web 
> server to generate traffic. I see the same basic type of thing happen 
> for other usage as well on my main network (ssh sessions, https/http 
> sessions, etc). When looking at tcpdump I am occasionally seeing (on isp0):
> 
> 19:35:27.591761 aa:aa:aa:5b:db:99 > bb:bb:bb:1f:33:da,
> 192.168.1.230.2542 > xx.xx.53.84.80: ., cksum 0xfade (correct), 
> 2295591733:2295591733(0) ack 167570634 win 0
> 
> If this packet was truly supposed to be going out on the external 
> interface, it should have gone through NAT and show a src ip of 
> 10.42.0.220. To make it more frustrating, even if I enable ifpfw at 
> layer 2, I am unable to capture these rogue packets. If I watch tcpdump 
> on net0 at the same time, I see the following:
> 
> 19:35:27.591767 aa:aa:aa:5b:db:98 > cc:cc:cc:10:04:ce,
> xx.xx.53.84.80 > 192.168.1.230.2542: ., cksum 0xfade (correct), 
> 913:913(0) ack 1256 win 0
> 
> The only other thing that I have noticed, is that the packets seem to 
> show up on the external interface at about the same time as the dynamic 
> rules expire. The dynamic rule would look like:
> 
> 192.168.1.230 2542 <-> xx.xx.53.84 80
> 
> Which is pretty much what I would expect. The same setup with a 
> non-stateful ipfw ruleset (using established keyword) doesn't seem to 
> have this problem. Any ideas? configuration follows.
> 
> 
> **** natd.conf ****
> unregistered_only yes
> dynamic yes
> #deny_incoming yes
> log_denied yes
> log_ipfw_denied yes
> 
> (deny_incoming was set, turned it off to see if it helped but it works 
> the same).
> 
> ***** ipfw.rules ****
> # Test stateful firewall + natd script
> cmd="/sbin/ipfw add"
> natout="skipto 1000"
> oif="isp0"
> iif="net0"
> inet="192.168.1.0/24"
> 
> NOROUTE="( 172.16.0.0/12 or 192.168.0.0/16 or \
> 0.0.0.0/8 or 169.254.0.0/16 or 192.0.2.0/24 or 224.0.0.0/4 or 
> 240.0.0.0/4 )"
> 
> ####
> # Start with a clean ruleset
> /sbin/ipfw -q -f flush
> 
> ####
> # Allow all traffic on the loopback and internal network, to keep this 
> simple.
> $cmd 2 allow all from any to any via lo0
> $cmd 5 allow all from any to any in via $iif
> $cmd 6 allow all from any to any out xmit $iif
> 
> # Translate incoming traffic here
> $cmd 200 divert natd ip from any to any in via $oif
> $cmd 205 check-state
> 
> # Outbound
> # Use stateful inspection to allow any connection from the internal 
> network.
> $cmd 300 $natout tcp from any to any out via $oif setup keep-state
> $cmd 305 $natout udp from any to any out via $oif keep-state
> $cmd 310 $natout icmp from any to any out via $oif keep-state
> 
> # Inbound
> # Prevent non-routable networks on the external interface.
> $cmd 400 deny all from $NOROUTE to any in via $oif
> 
> # Allow incoming DHCP for external network address assignment.
> $cmd 450 allow udp from any to any 68 in via $oif keep-state
> 
> # Allow incoming SSH to this machine
> $cmd 455 allow tcp from any to me 22 in via $oif setup keep-state
> 
> # Allow incoming ICMP
> $cmd 460 allow icmp from any to any icmptypes 0,3,11,12 in via $oif
> 
> $cmd 999 deny log ip from any to any
> 
> # NAT rule for outgoing traffic.
> $cmd 1000 divert natd ip from any to any out via $oif
> $cmd 1005 allow ip from any to any
> 
> Thanks for any insight,
> 
> -James