12.0-beta3 pf firewall NAT rule syntax for vnet jail using pf

[Ernie Luzar]

Kristof Provost wrote:
> On 9 Nov 2018, at 19:14, Ernie Luzar wrote:
> 
>     Hello lists;
> 
>     testing 12.0-beta3 vnet jail that is using pf firewall.
>     net.inet.ip.forwarding =1 for the vnet jail.
>     Host is running ipfilter firewall.
>     The kldload pf.ko pflog.ko command has been issued.
>     10.0.10.30 is the ip address assigned to the vnet jail in the jail.conf.
>     Using this nat rule
> 
>     nat on epair2b from 10.0.0.30/24 to any -> (vge0)
> 
> Is this rule set on the pf inside the jail?

YES

> 
>     vge0 is the hosts interface facing the public internet and a member
>     of bridge2 along with member epair2a.
> 
> Is this bridge on the host, so outside the jail?

YES
> 
> If so, how can the jail see the vge0 interface?

Through the bridge? I don't really know. Just guessing.
> 
> Best regards,
> Kristof
> 

I added pass to the pf nat rule so inbound packets that match entry in 
state table get passed automatically.

Now using this pf nat rule
nat pass on epair2b from 10.0.0.30/24 to any -> (epair2b)

This is the ifconfig -a on the host after the vnet jail is started.

em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0
         mtu 1500
	options=81249b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,
         VLAN_HWCSUM,LRO,WOL_MAGIC,VLAN_HWFILTER>
	ether d0:50:99:93:75:98
	inet 10.0.10.2 netmask 0xff000000 broadcast 10.255.255.255
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vge0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST>
         metric 0 mtu 1500
         options=3899<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,
         WOL_UCAST,WOL_MCAST,WOL_MAGIC>
	ether 00:16:36:4e:35:86
	hwaddr 10:00:60:21:00:93
	inet xx.xx.xx.xx netmask 0xfffff000 broadcast 255.255.255.255
	media: Ethernet autoselect (1000baseT <full-duplex,master>)
	status: active
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
	inet 127.0.0.1 netmask 0xff000000
	groups: lo
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
	groups: pflog
bridge2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0
         mtu 1500
	ether 02:5c:98:6f:9d:0a
	id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
	maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
	member: epair2a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 6 priority 128 path cost 2000
	member: vge0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 2 priority 128 path cost 20000
	groups: bridge
	nd6 options=1<PERFORMNUD>
epair2a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST>
         metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 02:d9:a3:a8:e7:0a
	inet6 fe80::d9:a3ff:fea8:e70a%epair2a prefixlen 64 scopeid 0x6
	groups: epair
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Here are the pf rules in the vnet jail

oif=epair2b
set block-policy drop
set fail-policy drop
set state-policy if-bound
scrub in on $oif all
set skip on lo0
nat pass on $oif inet from 10.0.0.30/24 to any -> ($oif)
block out log quick on $oif inet proto tcp from any to any port 43
pass log (all) on $oif
pass out quick on $oif all

I test vnet jail by issuing ping 8.8.8.8 and get "time to live exceeded" 
message. ping 10.0.10.2 get all lost packets normal message.

Is there some other way to test vnet jails from the host to verify they 
are working?

There will come a time when I will need to test vnet jails from the 
public internet. Its easy to enable ssh on the vnet jail and then use 
some other isp to ssh into the vnet jail. What would be the syntax of 
the remote ssh command to do this?

It's my understanding that vnet jails have their own network stack which 
means there is no interaction with the hosts network stack. Which also 
means there is no vnet firewall interaction with the hosts firewall. Is 
this correct?

Since I want all my vnet jails to access the public internet, can their 
epair just be added to a single bridge as another member or does each 
one need it's own bridge?

How is public internet traffic targeted to an individual vnet jail 
running on the host?

Thanks for your help on this.
Ernie Luzar