From nobody Wed Sep 18 21:34:16 2024 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4X8BkJ6Xxfz5Vg6f for ; Wed, 18 Sep 2024 21:34:32 +0000 (UTC) (envelope-from bchesneau@icloud.com) Received: from qs51p00im-qukt01080302.me.com (qs51p00im-qukt01080302.me.com [17.57.155.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4X8BkH5Qcvz523D for ; Wed, 18 Sep 2024 21:34:31 +0000 (UTC) (envelope-from bchesneau@icloud.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=icloud.com header.s=1a1hai header.b=sOZbLlwD; dmarc=pass (policy=quarantine) header.from=icloud.com; spf=pass (mx1.freebsd.org: domain of bchesneau@icloud.com designates 17.57.155.21 as permitted sender) smtp.mailfrom=bchesneau@icloud.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=icloud.com; s=1a1hai; t=1726695270; bh=Gr+1RvyNYWIP5rCRtANMlmLea5tQ8n7m/naUbjytka4=; h=From:Content-Type:Mime-Version:Subject:Message-Id:Date:To; b=sOZbLlwD9LfuNOX5PHapFIn6i8NTO7vd/MneaAiCquxA+JnVITxgWw04wLU7qBuRW JMUu/V/wL4wtmwvSbsGZruGHysdy/eQ4zn8R0cxiOiVTQG6I762CC9Tl1VOQ/xAZIO eqhJSO3VEfQoREZJNl+r141uhKBLFzKqegwSapajCZ1oIHd1O+gCtCh9BZP9TDVAXy Re2kZyZuKMLjD2yZ4KP2wR6Z7oYUAMEh77Y5dTtkE6zNcvADxeo/scajahpH/WrbuH P0U+DLlRb3Hk8QeofSlcdzTmRxbIZWUHXi3LP6SNmXzH/jPIao/vb2ecFbu25Tul2X xbzFwfIv8VPYg== Received: from smtpclient.apple (qs51p00im-dlb-asmtp-mailmevip.me.com [17.57.155.28]) by qs51p00im-qukt01080302.me.com (Postfix) with ESMTPSA id BFC951140196 for ; Wed, 18 Sep 2024 21:34:29 +0000 (UTC) From: Benoit Chesneau Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Subject: issue with ICMP with PF and nat and latest 14.1 Message-Id: <764EE8F1-BE88-4714-AD3F-9D93028FFEC4@icloud.com> Date: Wed, 18 Sep 2024 23:34:16 +0200 To: "freebsd-net@freebsd.org" X-Mailer: Apple Mail (2.3776.700.51) X-Proofpoint-GUID: eGhA6YDSShFuVo2iDG_NtdzDQ09ctS-J X-Proofpoint-ORIG-GUID: eGhA6YDSShFuVo2iDG_NtdzDQ09ctS-J X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-18_14,2024-09-18_01,2024-09-02_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 spamscore=0 adultscore=0 phishscore=0 malwarescore=0 mlxscore=0 mlxlogscore=999 clxscore=1011 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2308100000 definitions=main-2409180142 X-Spamd-Result: default: False [-5.28 / 15.00]; WHITELIST_SPF_DKIM(-3.00)[icloud.com:d:+,icloud.com:s:+]; NEURAL_HAM_SHORT(-0.85)[-0.854]; NEURAL_HAM_MEDIUM(-0.83)[-0.825]; SUBJECT_ENDS_SPACES(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[icloud.com,quarantine]; RWL_MAILSPIKE_VERYGOOD(-0.20)[17.57.155.21:from]; R_SPF_ALLOW(-0.20)[+ip4:17.57.155.0/24]; R_DKIM_ALLOW(-0.20)[icloud.com:s=1a1hai]; MIME_GOOD(-0.10)[text/plain]; ONCE_RECEIVED(0.10)[]; TO_DN_EQ_ADDR_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[icloud.com]; FREEMAIL_FROM(0.00)[icloud.com]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[17.57.155.21:from]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[icloud.com:+]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; APPLE_MAILER_COMMON(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:714, ipnet:17.57.155.0/24, country:US]; DWL_DNSWL_NONE(0.00)[icloud.com:dkim] X-Rspamd-Queue-Id: 4X8BkH5Qcvz523D X-Spamd-Bar: ----- Hi, It seems that since the latest update of PF in FreeBSD 14.1 mtr doesn't = provide a correct trace using the default. It works with the ``--udp` = and `-T` options so it doesn't seem to be an issue with the next hop. = Also mtr works perfectly on the firewall machine. Issue only happen on = the nated machines on the lan behind it. No issue with ipv6. I tried to change the config or pass everything but I still reproduce = the issue. Any idea to troubleshoot/fix it is welcome :) Eg of trace: ``` MacBook-Pro-de-Benoit-8.local (10.0.1.62) -> 1.1.1.1 (1.1.1.1) = = 2024-09-18T11:32:29+0200Keys: Help Display mode Restart statistics = Order of fields quit = Packets Pings Host = Loss% Snt Last Avg Best Wrst = StDev 1. 10.0.1.1 = 0.0% 11 4.7 6.4 3.6 22.6 = 5.7 2. (waiting for reply) 3. (waiting for reply) 4. (waiting for reply) 5. (waiting for reply) 6. one.one.one.one = 0.0% 10 6.8 6.6 5.6 11.7 = 1.8 ``` The configuration of PF is the following ``` table persist file "/etc/pf/lan.tbl" IP_OUT =3D "" ext_if =3D "vlan200" vlan_ifs =3D "{ vlan10, vlan20, vlan30, vlan31, vlan110, vlan120 }" # Macros set block-policy drop set skip on lo # Options scrub in all fragment reassemble # Normalize and reassemble fragmented = packets #scrub in all # nat nat from to ! -> $IP_OUT # Explicitly block unroutable addresses antispoof quick for ($ext_if) #pass proto icmp all # Drop invalid packets block in quick on $ext_if inet proto tcp all flags FUP/FUP # Dropping = invalid TCP packets block in quick on $ext_if inet proto tcp all flags S/SAFRUP # Dropping = weird flags # Allow all outgoing traffic from the internal network (LAN) pass out on $ext_if from any to any keep state # Allow incoming established and related connections (untracked) pass in on $ext_if proto tcp from any to any flags S/SA modulate state pass in on $ext_if proto { udp, icmp, icmp6 } from any to any keep state # Allow ICMP traffic for mtr (Echo Request, Echo Reply, Time Exceeded) pass in inet proto icmp icmp-type { echoreq, echorep, timex } keep state pass out inet proto icmp icmp-type { echoreq, echorep, timex } keep = state ``` I also tried a simpler version: ``` # Allow all outgoing traffic pass out on $ext_if all # Allow all incoming ICMP pass in inet proto icmp from any to any ``` While no errors, mtr on the lan still doesn't work. I have also tried to = log it : ``` pass in log proto icmp all ``` but no log appears. I am clue less right now. It seems the error is = related to `ICMP time exceeded in-transit` but I thought the issue would = be solved by the configuration below. What I'm missing? Beno=C3=AEt=