Issues putting jails on their own subnet

[Andrew Klaus]

Hello,

I'm trying to segregate some of my jails onto their own (DMZ) subnet.

Internal subnet: 10.0.3.0/24
DMZ subnet: 10.0.4.0/24

Both of these subnets are on my FreeBSD host, but I'm using a second
routing table for my DMZ jails as seen here:

---------------
setfib 1 netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            10.0.4.1           UGS         0  2393945  vlan4
10.0.3.0/24        link#12            U           0        0  vlan3
----------------

The problem I'm facing, is when I try to connect to the DMZ'd jail from the
10.0.3.0 network, traffic comes in on vlan4 like it's supposed to, but
replies back through on the vlan3 interface. I guess this makes sense,
because of that second route entry (that I can't override).

I've tried using PF to force the packets back through to 10.0.4.1, but it
doesn't seem to want to work.  Is the only other way to use the
experimental vnet/vimage?

Any ideas would be helpful.

Thanks,

Andrew