Multihomed system with jails routing issues

[Julian Elischer]

On 4/5/14, 10:22 AM, Chris Smith wrote:
> Hi All,
>
> I have a system with 1 network interface with 2 extra VLANs off it 
> and I'm having some trouble getting the routing working correctly 
> with it and jails.
>
> bge0     - management - 10.71.100.0/24
> bge0.101 - LAN        - 10.71.101.0/24
> bge0.103 - DMZ        - 10.71.101.0/24
>
> Here's what I want to achieve...
>
> Host:
> I want the host system to only listen on one interface, bge0. I want 
> NO ip addresses of the host on the vlan interfaces. The only service 
> it will be exposing is its sshd. The management address for this 
> system is 10.71.100.50.
>
Sounds to me that you want to use vimage jails.
check the vnet command to jail .


> Jails:
> The system will also host a variety of jails, each with an IP either 
> on the LAN or DMZ. I am using ezjail to manage the jails.
>
> Router:
> There is a router at the .254 address of every subnet that can route 
> between each network.
>
> I set up jail1 on bge0.101 with the IP 10.71.101.51. Since the host 
> does not have an address configured on bge0.101, I configured the 
> jail address as /24 instead of the default /32.
>
> My issues:
>
> * If I do not configure the jail as a /24 (e.g. /32), the LAN cannot 
> communicate with the jail.
>
> * When the jail is up and 10.71.101.51/24 is active, SSHing from the 
> LAN to the mgmt interface via the router fails, as the host tries to 
> send return traffic via the bge0.101 interface, even though traffic 
> arrived via the bge0 interface.
>
> So I did a whole lot of research for people having these apparently 
> problems, and decided to try the multiple routing table/fib 
> approach. So I recompiled my kernel, configured fib 1 with the LAN 
> interface route (setfib route add 10.71.101.0/24 -iface bge0.101), 
> set the jail fib and set the tunable net.addr_all_fibs = 0. I still 
> can't get this working correctly. ezjail still seems to add the 
> interface route to fib 0 by default (but it won't if i run ezjail 
> with the setfib 1 command).
>
> Using FIB 1 and trying to ping hosts on the LAN gives an error like: 
> sendto failed: invalid argument.
>
> Does anybody have any best practices for doing this, or anything 
> else I can try? I'm happy to share/pastebin any configuration and 
> I've tried most things I've found on the internet. I'm using FreeBSD 
> 10.0 with a custom kernel for multiple routing tables.
>
> Thanks in advance!
> Chris.
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>