Re: Tunnel interfaces and vnet boundary crossing

------- Original Message -------
On Tuesday, August 23rd, 2022 at 3:58 PM, Milan Obuch <freebsd-net@dino.sk> wrote:


> On Tue, 23 Aug 2022 15:06:00 +0000
> tt78347 tt78347@protonmail.com wrote:
> 
> > Hello. This is my first email to this mailing list, so I hope it has
> > been sent and formatted correctly.
> 
> 
> Well, top posting is being frowned upon a bit on mailing list, which
> has its racio, otherwise I think everything is fine :)
> 
> > Regarding tun interfaces, I do something similar to M. Gmelin,
> > however, I create the tunnel (tun, for openvpn) within the vnet jail.
> > I think your question is not about tun interfaces but about gre,
> > which I discuss near the end of this email.
> 
> 
> You are right, terminology is sometimes somewhat ambiguous. I have no
> issues with software tunnel interface (man tun). I did not find the
> right configuration for my needs for network tunnels (man gif, man gre,
> man if_ipsec, the last one could be the most desired one).
> 
> > Regarding devfs:
> > My /etc/devfs.rules contains the following ruleset, applicable to all
> > such jails:
> > 
> > [devfsrules_jail=5]
> > add include $devfsrules_hide_all
> > add include $devfsrules_unhide_basic
> > add include $devfsrules_unhide_login
> > add path bpf unhide
> > add path tun* unhide
> > add path bpf0 unhide
> > 
> > This ruleset ID is specified during jail creation. If you are
> > modifying this on a running system, I think you will need to use the
> > devfs utility. This is how I get tun interfaces into a jail, for use
> > with openvpn or otherwise.
> 
> 
> I use something similar in my setup, basically with expanded includes.
> 
> > General background:
> > All the vnet jails are to be connected to a bridge with epairs, and
> > NAT is to be performed for jail traffic exchanged with the
> > exterior.
> 
> 
> In general, epair is optional, in some use cases vnet jail could be
> decisively insulated from the rest of jails. But I know this one and
> use it if in accordance with my design goals.
OK. I just meant it to specify what was included in my suggested design.
> 
> > There is an additional vnet jail which handles ipsec; the ipsec jail
> > connects the host's jail network (usually a /24) to other local
> > networks. Each external local network's route is specified in the
> > host's routing table, with a next-hop being the ipsec jail. I
> > personally don't use ipsec interfaces, so in the ipsec jail the
> > traffic is extracted by the kernel (per TSs that are configured with
> > StrongSwan), encapsulated and sent out the host's external interface.
> > I have no knowledge of whether this is better or worse than using an
> > ipsec interface.
> 
> 
> This one is interested for me. Could you elaborate more on its design?
> Output of 'ifconfig', 'netstat -rn', 'pkg info', 'cat /etc/rc.conf'
> commands would be really usefull. If you do concern about privacy,
> maybe edit it a bit (I am not interested in exact IPs and such info).

I think I had misunderstood your original post, but here is information about this ipsec jail.
I have replaced certain info with variable names.

# cat /etc/rc.conf


devfs_load_rulesets="YES"

hostname="$HOSTNAME"
syslogd_enable="YES"
syslogd_flags="-s -v -v"

inetd_enable="NO"

defaultrouter="${HOST_NET}.1"

firewall_enable="YES"
firewall_script="/etc/ipfw.conf"
firewall_logif="YES"

sendmail_enable="NO" 
sendmail_submit_enable="NO" 
sendmail_outbound_enable="NO" 
sendmail_msp_queue_enable="NO"

# cat /etc/ipfw.conf
ipfw add allow ip from any to any
# cat /etc/rc.conf.local 
strongswan_enable="YES"
# netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            ${HOST_NET}.1       UGS     epair${JAIL_ID}b
127.0.0.1          link#1             UH          lo0
${HOST_NET}.0/24    link#${JAIL_ID}             U       epair${JAIL_ID}b
${HOST_NET}.${JAIL_ID}      link#${JAIL_ID}             UHS         lo0


This ipsec jail has epair IP address of ${HOST_NET}.${JAIL_ID}. 
For example, 192.168.1.100, where $JAIL_ID=100 and $HOST_NET=192.168.1
The host's bridge is at ${HOST_NET}.1
The only relevant package in this jail is security/strongswan.

In this design, all vnet jails have addresses on $HOST_NET using epair interfaces connected to one or more virtual host bridge interfaces.
ARP takes place on this bridge, and the host forwards packets between jails.
The ipsec jail is just one of those jails. 
The host also has routes for other sites' local networks. These routes are set to go through the ipsec jail.
The ipsec jail then encapsulates them, and from the host's perspective, when it leaves the ipsec jail, it is just ESP/UDP.

In this design, ipsec is used for site-to-site tunneling, and GRE is used for use-case-specific tunneling between hosts on this WAN. A common use case for this is when a machine needs to set a route or a default route through a gateway that is not on a connected subnet. Operating ipsec and GRE independently means you can use GRE where necessary, with the whole WAN being encapsulated with ipsec. Traffic within a host being unencrypted.

> 
> > Specific steps inside an openvpn jail:
> > 
> > After an openvpn jail is started, the tunnel (tun interface) is
> > created in the jail by running the openvpn daemon. I think tunnel
> > numbers (appearing on the interface name) need to be globally
> > distinct. I'm not sure if you needed any assistance with this.
> 
> 
> In my scenario, I use tun devices pre-created before running OpenVPN
> process. This allows for better control and easier to understand setup.

I think I misunderstood your posts. I thought you were running openvpn clients, but I see from reading again that you are running openvpn servers. I have never run an openvpn server before.

> 
> > Regarding gre interfaces, I also create these within the jail. I have
> > not ever had any problems with this. I don't know that a specific
> > device is needed, so I don't know if devfs is involved here.
> 
> 
> This setup just requires connection to outer network, which I would
> like specifically to be avoided as it makes setup a bit more
> complicated. Just a reminder, I have almost twenty of running OpenVPN
> instances in one jail, with almost twenty vnet jails handling the
> routing for each VPN.

I don't fully understand what is happening. What do you mean by outer network, and why would it be an issue to create the GRE tunnel inside the jail, instead of creating it outside the jail and moving it into the jail?

I think you have your openvpn server instances in a vnet jail, and then child jails within that jail which are doing something related to routing. Are you using many fwd-type firewall rules, instead of using default routes?
Would it make sense to have each openvpn server in a separate vnet jail, each with its own GRE interface, and do away with your child vnet jails that handle routing?


Sorry for my misunderstandings.

> 
> [ snip ]
> 
> I hope this time I made it a bit clearer what I am looking for.
> 
> Regards,
> Milan