IPFW with NAT : Problems with duplicate packets on FreeBSD 10.3-RC3

[Dr Josef Karthauser]

> On 8 Apr 2016, at 00:11, Dr Josef Karthauser <joe at truespeed.com> wrote:
> 
>> On 7 Apr 2016, at 17:08, Dr Josef Karthauser <joe at truespeed.com <mailto:joe at truespeed.com>> wrote:
>> 
>> Looks like the first packet is being retransmitted, which means that the nat is probably misconfigured and the TCP connection is broken in some strange way.
>> 
>> Does anyone have a clue as to where to look? The ipfw rules are simple enough - what have I missed?
> 
> Ok, the packet definitely isn’t being retransmitted. I’ve done a tcpdump/pcap capture and taken a look and I get a packet that I’ve included below.
> 
> It’s got a 'HTTP/1.1 200 OK’ inserted mid-flow right in the middle of an HTTP response. Looking at this I’d be inclined to think it’s a bug in the webserver/tomcat, however, what’s strange is that if I ‘curl' the jailed web server directly from the host machine on the private IP address (bypassing the NAT), the HTTP response  received is perfectly fine. It’s only when I do an HTTP request to the public IP address and go through the NAT that I experience the problem.
> 
> How could this happen? Is it a buggy packet reassembly in the kernel perhaps?
> 

Adding: "ipfw add reass all from any to any” to the beginning of the ipfw rule set doesn’t make any difference to the behaviour. 

Joe