From nobody Tue Oct 11 11:08:57 2022 X-Original-To: pf@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MmtMJ1SDzz4fW1W for ; Tue, 11 Oct 2022 11:09:00 +0000 (UTC) (envelope-from infoomatic@gmx.at) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MmtMH1Sl4z3Tnh for ; Tue, 11 Oct 2022 11:08:59 +0000 (UTC) (envelope-from infoomatic@gmx.at) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1665486538; bh=Lkt71vuCZwYQA4vXmR09MlGWooJ6KVosF29D17/yjhk=; h=X-UI-Sender-Class:Date:Subject:To:References:From:In-Reply-To; b=jw8r3F2mb2rW9cx8DlMW1PiVwIulaYD2SCYpIIOvImUp4xHEMoFjU0K1uvgz/oAJw 6c1VgYtBL9J9a0Xf+DdLxymTgypUkT7iZ0T4qeDO6/YqoMbTjVQ8ItavO+xr3QVik8 tn0PLXaLrI8YVKX0fs83LsYEbAWP4VWkHLItF3mE= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [10.0.1.209] ([178.114.236.102]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MnakX-1pP08q3PQD-00jXVY for ; Tue, 11 Oct 2022 13:08:57 +0200 Content-Type: multipart/alternative; boundary="------------0glmfJ3re0Wcpg5eEbGFHgDL" Message-ID: Date: Tue, 11 Oct 2022 13:08:57 +0200 List-Id: Technical discussion and general questions about packet filter (pf) List-Archive: https://lists.freebsd.org/archives/freebsd-pf List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-pf@freebsd.org X-BeenThere: freebsd-pf@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.3.0 Subject: Re: PF: nat on ipsec Content-Language: en-US To: pf@freebsd.org References: <1ba3e340-e204-15b0-d395-a942c97c39f5@gmx.at> From: infoomatic In-Reply-To: X-Provags-ID: V03:K1:4rBa7C+AyskZlhVcAA9yHLmFJoqBP57LjbKPbpCkZ9i7SZOGRn/ xZ2xWn13q9YVBagcxg3dWJVpTG3d1G294LyWHND+lrLWPJQjMp4/DwYqg9yHNb/jzwNRChB H21YCZVMWr9mV6q2+2VtI9/HnQM0VbBbSUhMPyvw7y3tFCEvma2cNBbGfWEthBA/3AVXAEU zbFcH3/d9WVvbZNZRpz2w== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:lA9/7epuaug=:c9NiosX04iSbq0BCwGRQQQ xLNvvOBGtYdf3cQyGjy5mZvpyvNxHw6piR73Cg2tDRv15NwvY7FrrElYXLaGdk0oo32WFYl8u 7gWtOtk4yvNZ96vQ56JEMOPtRWjN42cWd3an9caMlkbjqG/qOlRk8ULRotuPP1kLWq6J8TWjR LrQIJTNPSYyZEBUs+4TCskCN+TXYLsXj8QcpXN+jq9GBszZOgF6zNMauLZHWFoh3C+01/vham 9x/HjS0ceheIp0ab8esmjwzU8IOa6spheT6XjxOapuytnfTpzr52UwOB0iG21MVRvDM+UMzsO x1SPRIJqY7cVh/ZMuDJQ3F8rKz5d4dPkc37VTIdH590x0y/Ppy1DWo6pyJo3h8xsMehzWf8UP DAcaqYINEl5TOM3XPM0JwfP7XXu4zuuZhy+OXuXka1PdIncXGsjEeoEkSl0TKTJp19+BRbMY8 GBWTpkDe/bvLaDY8f0FJyP2hrdWpjpAQevetAmjTI0XKotQsZ6hMXswJcO3Sm2xt2bIHPkRrs Jzo6znHhstXM0lF2rj9C2HFC4/xAdY0f7BdLFai44qZdj75qOXAhkAEs1fh/oYoXXv2tDaCXx mfAV3BBvOElHovVwNysFN3j8eC1bAzrRvv+zyOhn6AiRpgOe9tFjL0ijpkdMpDlUtc3ino746 Aa5OyrIlvDVqG15hTBsUSr45bOVQwYEQhSvDNxaqazY7C/Rt19I39oK8xWmmXIbjty/0aFOf/ UObq38I0cCSz8uceW7GgMFWk6dIKY/zA7LmeihjK5ZSMcpANIOJ+z7C7aOFO4f87N+LOt+Zmk qm604N6aUdByGm+2m7j7g7TnNuRzz1uR0m0IfVPJS2W7lV0AgWDI4FpNLDgk120ycnimNWxpu 9A1IW/m4/ynxTNoyXVJVt5eR4zQOEceldGQDtBfHyc8t9nV99UyTceEpr1Tr1NHv/FjAkYqaU vxM+mhfDmX4tlXjiH/w5q/O+Hpc6XfjvvacdMkYQ3e6SEsuZFF7VVNCuBfxJkKhmwF2j/rGzo PPwijv9kK0KXHrv6CC3kMn7o/44RwPEGe7B6+xH1f0V3cLXG8PMc0DSaiGA4gVd9VnTgCSKIm d0fNG2mc1c/4XyNszGaBqpN3VSIsFvrhJdEujHvJmxj60gEf0FFTJvbpgBaBEMjA9me9u1K5g WHle56EsMC2A4Cc4JE5oJ0z8pL X-Rspamd-Queue-Id: 4MmtMH1Sl4z3Tnh X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmx.net header.s=badeba3b8450 header.b=jw8r3F2m; dmarc=pass (policy=none) header.from=gmx.at; spf=pass (mx1.freebsd.org: domain of infoomatic@gmx.at designates 212.227.17.20 as permitted sender) smtp.mailfrom=infoomatic@gmx.at X-Spamd-Result: default: False [-4.08 / 15.00]; DWL_DNSWL_LOW(-1.00)[gmx.net:dkim]; URI_COUNT_ODD(1.00)[5]; NEURAL_HAM_SHORT(-1.00)[-0.998]; NEURAL_HAM_LONG(-1.00)[-0.997]; NEURAL_HAM_MEDIUM(-0.98)[-0.984]; DMARC_POLICY_ALLOW(-0.50)[gmx.at,none]; R_DKIM_ALLOW(-0.20)[gmx.net:s=badeba3b8450]; R_SPF_ALLOW(-0.20)[+ip4:212.227.17.0/27]; RCVD_IN_DNSWL_LOW(-0.10)[212.227.17.20:from]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MLMMJ_DEST(0.00)[pf@freebsd.org]; PREVIOUSLY_DELIVERED(0.00)[pf@freebsd.org]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; DKIM_TRACE(0.00)[gmx.net:+]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[gmx.at]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmx.at]; MIME_TRACE(0.00)[0:+,1:+,2:~]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N This is a multi-part message in MIME format. --------------0glmfJ3re0Wcpg5eEbGFHgDL Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable > IPsec traffic flow is complicated. Have a look at enc. It's been > instrumental in helping me fix this class of issue in several > instances. > YMMV. > > https://www.freebsd.org/cgi/man.cgi?query=3Denc&sektion=3D4 > > I have no clue why the host should try to do anything with the packets except for changing source and destination address (NAT). The tunnel is setup between AWS and the VM on the host. The ssh connection from AWS to a client "behind" opnsense works. However, as soon as I try to make a ssh connection from the jail ("behind" opnsense) to AWS, the packets from my local vpn endpoint (opnsense VM) do not get NATed on the host. The host just tries to forward those UDP Port 4500 packets with the private ipv4 address of the opnsense VM as source on the egress interface with the public interface. This of course should not happen. Routing problems can be ruled out, the exact same configuration is working on a Linux host hosting the same opnsense VM. A simple |sysctl net.ipv4.ip_forward=3D1 && iptables -t nat -A POSTROUTING --source 192.168.251.100 -j SNAT --to-source $public_vpn_ip| did the trick. There is a strange problem here, maybe it is not pf related, maybe something in the stack interferes with those packets. Anyone knows/could guess if this works with ipfw instead? --------------0glmfJ3re0Wcpg5eEbGFHgDL Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit
IPsec traffic flow is complicated. Have a look at enc. It's been
instrumental in helping me fix this class of issue in several instances.
YMMV.

https://www.freebsd.org/cgi/man.cgi?query=enc&sektion=4


I have no clue why the host should try to do anything with the packets except for changing source and destination address (NAT).

The tunnel is setup between AWS and the VM on the host. The ssh connection from AWS to a client "behind" opnsense works.

However, as soon as I try to make a ssh connection from the jail ("behind" opnsense) to AWS, the packets from my local vpn endpoint (opnsense VM) do not get NATed on the host. The host just tries to forward those UDP Port 4500 packets with the private ipv4 address of the opnsense VM as source on the egress interface with the public interface. This of course should not happen. Routing problems can be ruled out, the exact same configuration is working on a Linux host hosting the same opnsense VM. A simple

sysctl net.ipv4.ip_forward=1 && iptables -t nat -A POSTROUTING --source 192.168.251.100 -j SNAT --to-source $public_vpn_ip

did the trick. There is a strange problem here, maybe it is not pf related, maybe something in the stack interferes with those packets.

Anyone knows/could guess if this works with ipfw instead?

--------------0glmfJ3re0Wcpg5eEbGFHgDL--