Wierd tunnel+MTU issue

[Andre Oppermann]

Aragon Gouveia wrote:
> 
> Hi,
> 
> No, it's not that.  No filtering is taking place.  I've figured out the
> problem, but I'm not sure how to solve it.  Here's what I think is the
> problem.
> 
> >From a tcpdump transcript:
> 
> 09:56:37.652907 <host>.4185 > <notebook>.80: S 487952620:487952620(0) win 57344 <mss 1460,nop,wscale 0,nop,nop,timestamp 611364250 0> (DF) [tos 0x10]
> 09:56:37.653076 <notebook>.80 > <host>.4185: S 4069940133:4069940133(0) ack 487952621 win 65535 <mss 1452,nop,wscale 1,nop,nop,timestamp 317780676 611364250> (DF)
> 
> <notebook> is my notebook running Apache.  As can be seen above, it's
> negotiating an MSS of 1452 with the peer, which it should not be doing.  The
> reason it's doing that is because my default route is via an interface with
> an MTU of 1492 - the tun interface opened by userland ppp for the PPPoE
> session over my ADSL bridge.
> 
> As I said, I'm using ipfw fwd to source route packets from <notebook>
> (the vtun tunnel interface address) to the vtun tunnel's remote end-point.
> But I'm guessing MSS is chosen based on the host's routing table.  Which
> makes perfect sense.
> 
> So to prove my suspicion I added a route on my notebook as follows:
> 
> route add -host <host> 196.15.a.y
> 
> 196.15.a.y being the vtun tunnel's remote end-point.
> 
> Now the tcpdump transcript looks like this:
> 
> 10:10:21.227506 <host>.2404 > <notebook>.80: S 996010957:996010957(0) win 57344 <mss 1460,nop,wscale 0,nop,nop,timestamp 3560266528 0> (DF) [tos 0x10]
> 10:10:21.227717 <notebook>.80 > <host>.2404: S 2935622965:2935622965(0) ack 996010958 win 65535 <mss 1204,nop,wscale 1,nop,nop,timestamp 318604232 3560266528> (DF)
> 
> The tunnel's interface MTU was set at 1256 when I did this.  So the
> negotiated MSS is now correct and things are working.
> 
> But I need to be able to route based on source address and ipfw fwd is the
> only way I know how to do it.  Can anyone think of a clever workaround for
> this?  Is there a way to force the TCP stack to use a set MSS regardless of
> what the routing table and interface MTU say?

You are onto something.  It seems tcp_output() doesn't handle the error
cases it gets from ip_output() all too well these days.  I suspect this
is the same problem we have in kern/71184.  I'll look into it later today.

Could you please file a PR with all information you have provided so far
and your observations etc.  Just merge your emails together and submit it
as text.  Then give me the PR number so I can take it over.

-- 
Andre