Running jails on multiple subnets with multiple interfaces

[Josh Paetzel]

Jeffrey Williams wrote:
> I have a server with two interfaces, I want to run the host and a couple of 
> jails using one interface on one subnet (internal interface, private IP, behind 
> NAT/firewall) and some other jails using the other interface on another subnet 
> (external interface, public IP, DMZ).
> 
> Now my understanding of the challenge in doing this, is that the network stack 
> is not "virtualized" in the jails, so all the jails use the same routing table, 
> and for obvious reasons only one default router. (also just for sake of clarity 
> I don't want to enable routing between interfaces on the jail host)
> 
> Now if I understand all this correctly, then what will happen is, if I set the 
> default router to the internal networks exit router (the NAT/firewall), then 
> the jails listening on the external interface will only be able to talk to 
> their local subnet, and because the internal subnet won't exist for them they 
> won't be able to connect to the network at large.
> 
> If I set the default router to the external networks exit router (the DMZ 
> perimeter firewall) then the host and jails listening on the internal network 
> won't be able to be able to talk to the internet beyond the local nets, the 
> jails because the external network doesn't exist for them, and the host because 
> even though it can talk to both nets, the services are configured to only 
> listen to the internal net, and the it will be trying to send all outgoing 
> traffic to the public net, thus not creating and NAT table entries on the 
> NAT/Firewall for the return connections.
> 
> Is there anyway to achieve what I have trying to do?
> 
> Thanks
> Jeffrey williams

PF makes a very effective workaround to this with it's route-to
option...effectively letting you bypass the routing table altogether
and set up per IP behavior.

For instance, I use it in the following scenario, where a box has two
interfaces with public IPs and I don't want answers to connections on
the 'secondary' interface to go out the default route.

connection 1's router 192.168.1.1
em0 ip 192.168.1.2/24

connection 2's router 10.0.0.1
em1 ip 10.0.0.2/24

if connection 1 is the 'primary' link then set the default route to
192.168.1.1 and put the following rule in pf.conf

pass out route-to (em1 10.0.0.1) from 10.0.0.2 to ! 10.0.0.0/24

If you were to give more concrete examples of your config I could
probably help you out with a workable pf solution.

-- 
Thanks,

Josh Paetzel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-net/attachments/20070828/39e0239c/attachment.pgp