IPSEC traffic doesn't work realiably after upgrading from 4.11 to 5.4

[Ari Suutari]

(replying to myself again...)


Ari Suutari wrote:
> Some more information to this one: This seems to be some kind of
> odd routing problem. I just recreated the setup under vmware and noticed
> that when the problem occurs the outgoing ESP packets are
> flowing on interface that has the default route (em0), not
> on tun0. The routing table entry looks correct (ie. points
> to tun0), but ESP packets don't seem to obey it
> (until setkey -F is issued).

	There seems to be a field called sa_route, which
	caches routing information in SA if I understood
	the code correctly. What happens here is this:

	- when tun0 goes down, the code in ipsec.c notices
	that route is not up any more and gets a new route for
	packet. In this case it gets the default route.
	- when tun0 goes up again, ESP packets are still
	sent to default route, because it is still valid.
	- setkey -F clears this cached information, restoring
	correct operation.

	I coudn't find any place where sa_route stuff is invalidated
	when routing table changes. If so, isn't this kind of a serious	
	problem ?

		Ari S.

> 
>     Ari S.
> 
> Ari Suutari wrote:
> 
>> Hi,
>>
>> I have upgraded a vpn server from FreeBSD 4.11 to 5.4-RELEASE.
>> The box as about 20 vpn connections to other FreeBSD machines,
>> the physical connection is via tun0 ... tun20 devices.
>>
>> Traffic flow is something like this:
>>
>> my internal net ->
>>     vpn server em1 ->
>>     vpn server ipsec ->
>>     vpn server tun0 ->
>>     vpn server em0 ->
>>     internet ->
>>     remote freebsd fxp0 ->
>>     remote freebsd tun0 ->
>>     remote freebsd ipsec ->
>>     remote net
>>
>> Remote FreeBSD box is still running 4.11.
>> Ipsec is the kame version, not FAST_IPSEC.
>>
>> (tun0 stuff is created by vtun software, which is used
>> to get around various restrictions, like ISP providing
>> private addresses only).
>>
>> This has been working very well for years under FreeBSD 4.x.
>>
>> After upgrading to 5.4, things seem to work at first. However,
>> when physical connection has problems, causing tun0 device to
>> go temporarily down on server the vpn never recovers from it.
>> When tun0 comes up back again, IPsec SAs seem to be valid
>> on both sides. Non-ipsec traffic works without problems
>> over tun0 as does *incoming* ipsec traffic from remote
>> FreeBSD box. Outgoing ipsec packets seem to vanish completely.
>>
>> It seems that the problem can also be triggered by running
>> ifconfig tun0 down && ifconfig tun0 up.
>>
>> netstat -s -p ipsec doesn't show any errors. To recover
>> from situation, issuing setkey -F to flush all SAs helps.
>> Flushing only the SAs related to this connection does not help,
>> neither does removing related policies and adding them again.
>>
>> I would'n like to go back to 4.x series, so I'm looking
>> for fix/workaround for this.
>>
>>     Ari S.
>> _______________________________________________
>> freebsd-net at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
> 
>