IPv6 and NAT

From: Goran Mekić <meka_at_tilda.center>
Date: Mon, 07 Feb 2022 09:41:54 UTC
Hello,

I'm trying to configure jails on desktop to have local addresses with
the prefix of fd1a:db86:3f72:9dc4:: and NAT them using PF. What I
observe is that when I try "ping -6 google.com" my router receives ICMP
request from address with "fd1a:db86:3f72:9dc4::" as prefix.

Inside jail (eth0 is epair):
eth0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether 00:a0:98:a6:60:8d
	hwaddr 02:1e:d3:64:68:0b
	inet 172.16.0.10 netmask 0xffffff00 broadcast 172.16.0.255
	inet6 fe80::2a0:98ff:fea6:608d%eth0 prefixlen 64 scopeid 0x3
	inet6 fd1a:db86:3f72:9dc4:2a0:98ff:fea6:608d prefixlen 64 autoconf
	groups: epair
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
	nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>

netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            172.16.0.254       UGS        eth0
127.0.0.1          link#1             UH          lo0
172.16.0.0/24      link#3             U          eth0
172.16.0.10        link#3             UHS         lo0

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
default                           fe80::5a9c:fcff:fe10:ff90%eth0 UG        eth0
::1                               link#1                        UHS         lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
fd1a:db86:3f72:9dc4::/64          link#3                        U          eth0
fd1a:db86:3f72:9dc4:2a0:98ff:fea6:608d link#3                   UHS         lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%lo0/64                     link#1                        U           lo0
fe80::1%lo0                       link#1                        UHS         lo0
fe80::%eth0/64                    link#3                        U          eth0
fe80::2a0:98ff:fea6:608d%eth0     link#3                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0


On the host (cbsd0 is bridge, epair1a is host end of eth0 in the jail):
ifconfig cbsd0
cbsd0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: re0
	ether 58:9c:fc:10:ff:90
	inet 172.16.0.254 netmask 0xffffff00 broadcast 172.16.0.255
	inet 172.16.1.254 netmask 0xffffff00 broadcast 172.16.1.255
	inet 172.16.0.253 netmask 0xffffffff broadcast 172.16.0.253
	inet6 fe80::5a9c:fcff:fe10:ff90%cbsd0 prefixlen 64 scopeid 0x3
	inet6 fd10:6c79:8ae5:8b91::1 prefixlen 64
	inet6 fd10:6c79:8ae5:8b91::2 prefixlen 128
	id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
	maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
	member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 5 priority 128 path cost 2000
	groups: bridge
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

ifconfig re0
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
	ether bc:ae:c5:e1:31:d0
	inet6 fe80::beae:c5ff:fee1:31d0%re0 prefixlen 64 scopeid 0x1
	inet6 2001:470:1f1a:2c6:beae:c5ff:fee1:31d0 prefixlen 64 autoconf
	inet 192.168.111.3 netmask 0xffffff00 broadcast 192.168.111.255
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>

pfctl -sn
nat on re0 inet from <cbsd> to any -> (re0) round-robin
nat on re0 inet6 from (cbsd0:network) to any -> (re0:0)
rdr-anchor "cbsd/*" on re0 all

For completeness, these are my rules:
# Macros and tables
ext_if = "re0"
# SyncThing example
dhcp_ports = "{ bootps, bootpc }"
tcp_ports = "{ 3000, 7681, 22000}"
udp_ports = "{ 21027 }"
table <cbsd> persist

# Options
set block-policy drop
set skip on lo0

# Normalization
scrub in all

# NAT
nat on $ext_if inet from <cbsd> to any -> ($ext_if)
nat on $ext_if inet6 from (cbsd0:network) to any -> ($ext_if:0)
rdr-anchor "cbsd/*" on $ext_if

# Quick rules
antispoof quick log for ($ext_if)

# Rules
block in log on $ext_if
pass out
pass proto tcp to any port ssh
pass proto { icmp, igmp, icmp6 }
pass in proto tcp from any to any port $tcp_ports
pass in proto udp from any to any port $udp_ports
pass in proto udp from any to any port $dhcp_ports



I can see this on cbsd0 and re0 interfaces:
tcpdump -n -i cbsd0 ip6
10:37:08.473385 IP6 fd1a:db86:3f72:9dc4:2a0:98ff:fea6:608d > 2a00:1450:400d:80d::200e: ICMP6, echo request, seq 0, length 16


It is what I expect, as NAT is yet to be done, but then I don't expect
that source address to be visible on re0:
tcpdump -n -i re0 ip6
10:38:27.580473 IP6 fd1a:db86:3f72:9dc4:2a0:98ff:fea6:608d > 2a00:1450:400d:80d::200e: ICMP6, echo request, seq 0, length 16


I am just started learning about IPv6 so if anything is obviously wrong,
please point me to the docs and I'll happily RTFM. At this point I have
no clue what I did wrong.

Thank you,
meka