Multi-homing, jails, and source address selection

[Stefan Bethke]

I'm having some trouble configuring a dual-homed jail host, running - 
current from about 4 weeks ago.

My machine has one external interface em0 connected to an /27 IPv4  
network. Additionally, I have a VPN interface tun0 provided by an  
OpenVPN instance with a private /18 range.

I'd like my jails to be dual-homed, with a public and a VPN address  
each. Processes in the jail should pick the appropriate source address  
depending on the destination address, so that the source address for a  
connection going to a VPN address will be the jails' VPN address, and  
all other connections will use the jails' public IP.

I have a couple of questions that I can't seem to find answers to:

How do I get the VPN addresses configured? tun0 won't accept them  
(since ptp interfaces require a destination address).  If I use lo0, I  
seem to have source address selection issues.  I've experimented with  
various setups, but haven't found one that would work just right.  In  
the example below, if I ping from foo to a VPN address, the source  
address is foo's public IP.  If I run ping with -S10.0.63.3, the  
source address still is 192.0.2.3.

Is there any documentation on how source addresses are selected? I  
thought I remembered that on unbound sockets the destination route  
would be used to pick the first address of the outgoing interface as  
the source address; the same address would be picked on connecting a  
socket.

I'm currently running with this configuration in rc.conf:

cloned_interfaces="tun0"
ifconfig_em0="192.0.2.2/27"
ifconfig_tun0="10.0.63.1 10.0.63.255"

defaultrouter="192.0.2.1"
inetd_flags="-wW -a 192.0.2.2"
static_routes="openvpn"
route_openvpn="10.0.0.0/18 10.0.63.255"

jail_enable="YES"
jail_set_hostname_allow="NO"
jail_sysvipc_allow="YES"
jail_devfs_enable="YES"
jail_mount_enable="YES"

jail_list="foo bar baz"
jail_foo_rootdir="/jail/foo.example.com"
jail_foo_hostname="foo.example.com"
jail_foo_ip="em0|192.0.2.3,lo0|10.0.63.3"


Any suggestions?

-- 
Stefan Bethke <stb at lassitu.de>   Fon +49 151 14070811