Packetloss due to ipfw + kernel NAT?

[Terrence Koeman]

I was troubleshooting an intermittent network connectivity problem, and I noticed something weird.

My situation:

[internet]<->[freebsd box]<->[clients]

FreeBSD box (9-STABLE) has 172.16.0.1 on int0 (mtu 1500), x.x.172.84-85 on ng0 (pppoe via mpd, mtu 1492). Clients are assigned from 172.16.10/24{100-200}.

I stripped almost everything from my ruleset, so this remains:

natip="x.x.172.85"
$cmd enable one_pass
$cmd nat   10 config ip ${natip} same_ports
$cmd add 04020 nat 10 all from any       to ${natip}      in
$cmd add 04031 nat 10 all from ${intnet} to not ${intnet} out

Now, I suspected a MTU issue, so I tried some different packet sizes to see what happens:

On FreeBSD box:

ping -S x.x.172.84 -s 1400 mediamonks.net -> no packetloss
ping -S x.x.172.84 -s 1500 mediamonks.net -> no packetloss
ping -S x.x.172.84 -s 2500 mediamonks.net -> no packetloss
ping -S x.x.172.84 -s 3000 mediamonks.net -> no packetloss
ping -S x.x.172.84 -s 5000 mediamonks.net -> no packetloss

ping -S x.x.172.85 -s 1400 mediamonks.net -> no packetloss
ping -S x.x.172.85 -s 1500 mediamonks.net -> ~40% packetloss
ping -S x.x.172.85 -s 2500 mediamonks.net -> ~40% packetloss
ping -S x.x.172.85 -s 3000 mediamonks.net -> ~3% packetloss
ping -S x.x.172.85 -s 5000 mediamonks.net -> no packetloss

On client 172.16.10.101 (Windows 7 x64):

ping -l 1400 mediamonks.net -> no packetloss
ping -l 1500 mediamonks.net -> ~40% packetloss
ping -l 2500 mediamonks.net -> ~40% packetloss
ping -l 3000 mediamonks.net -> no packetloss
ping -l 5000 mediamonks.net -> no packetloss

If I set natip to x.x.172.84 the packetloss moves to that IP and remains the same for the client. Forcing the MTU on the Windows client to 1492 does not change the result. I double checked the result for packetsize 3000 since the result differs between the client and the FreeBSD box, but there is really no packetloss for the client while there is some on the FreeBSD box.

Does someone know what is happening here? Is this a bug in ipfw?

-- 
Regards,
T. Koeman, MTh/BSc/BPsy; Technical Monk

MediaMonks B.V. (www.mediamonks.com)
Please quote relevant replies in correspondence.