From nobody Thu Sep 09 18:02:18 2021 X-Original-To: freebsd-pf@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C339217BD7FC; Thu, 9 Sep 2021 18:01:55 +0000 (UTC) (envelope-from kisscoolandthegangbang@hotmail.fr) Received: from EUR06-VI1-obe.outbound.protection.outlook.com (mail-vi1eur06olkn20830.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe11::830]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H56Jy3nJ0z4pQy; Thu, 9 Sep 2021 18:01:54 +0000 (UTC) (envelope-from kisscoolandthegangbang@hotmail.fr) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nOx+Rkhz7R0kHy14CsISzQ1sFa/0lzMvz4LM4RIFFS2p0DfPPPJhDv7S2M7EY53uAe+i1G6NXbw9c/MvDN4SXIeAOUa0H7/a7DV/J03RmD4rWHkJ4F6GGv8IC11Akak+a2oAzL0owPDJAI2/DD5tk4SQ75iWyZ89X/cQ8L5Wf18CEM+jsXwU/Hq6NANQbZJJyaoJvcDAg6/4AnFRaNYlb85F5PL1IDQkfEJ1qcNhdYibCbCgmU80l0yIIX516oL9Z+Og2SbnRLfOdF48O7A4hs1WPYPPzxbF7KkRr41XFPUXlg3+DP1fmWUXvElrkiu8AYzRUrk0dykPQ35/anAc6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=BMKeYOg+4UcwfJ3+YrwFU83RUYvP4qcUtKBUMMzZ+Lk=; b=f7vU6umW5z6Aei0X2vKca92+m42+qgmao4HIFuVtWrFtUrOXuoXUN+zb4basGYLZiCkFAvWnBwQCplEkfvkkZViOgTt1aPMFo+XMJUu1i9YwXlaN6Xoal+sjln+XTm6R2WS7vvQlmqLUsYrbCaHD+o9wLZScQGEvEdp7oo87d2TNXoknaZVXplqKSgXGqvnGT4N1Xq7a3+X4jqN8rPDKGgm6SNz3DApwOuXM0n5WPT19Lvjg1culOsckOhuItdqUfG3qYloYazSNkVoCyJ2iNSoHHYhX5jb6QB0h6Nvf7OMqXc1OpkGlSuNY76/txRABtVAKbFOsIhkjxognkdJAdw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none Received: from VI1EUR06FT014.eop-eur06.prod.protection.outlook.com (2a01:111:e400:fc37::42) by VI1EUR06HT075.eop-eur06.prod.protection.outlook.com (2a01:111:e400:fc37::504) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14; Thu, 9 Sep 2021 18:01:46 +0000 Received: from AM9PR07MB7956.eurprd07.prod.outlook.com (2a01:111:e400:fc37::41) by VI1EUR06FT014.mail.protection.outlook.com (2a01:111:e400:fc37::323) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Thu, 9 Sep 2021 18:01:46 +0000 X-IncomingTopHeaderMarker: OriginalChecksum:39E3D70EA4F3417865064D24DF3FA795E8455EBFF6FE78FF35F8C688728E61EA;UpperCasedChecksum:A93A69FBAE978519943EA45CE650CDC65F45A16240B7FD5BC1AC074A518058C2;SizeAsReceived:7600;Count:45 Received: from AM9PR07MB7956.eurprd07.prod.outlook.com ([fe80::87e:4c4b:980e:2cdd]) by AM9PR07MB7956.eurprd07.prod.outlook.com ([fe80::87e:4c4b:980e:2cdd%9]) with mapi id 15.20.4500.015; Thu, 9 Sep 2021 18:01:46 +0000 Date: Thu, 9 Sep 2021 20:02:18 +0200 From: kaycee gb To: freebsd-net@freebsd.org, freebsd-pf@freebsd.org Subject: Issue with packets routing/forwarding Message-ID: X-Mailer: Claws Mail 3.17.6 (GTK+ 2.24.31; x86_64-slackware-linux-gnu) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TMN: [Vh2wmkvPdgcnMeUINtDgSB/Ychl0hkws] X-ClientProxiedBy: MR2P264CA0039.FRAP264.PROD.OUTLOOK.COM (2603:10a6:500::27) To AM9PR07MB7956.eurprd07.prod.outlook.com (2603:10a6:20b:30d::20) X-Microsoft-Original-Message-ID: <20210909200218.1257d1e8@slackstro.home.lan> List-Id: Technical discussion and general questions about packet filter (pf) List-Archive: https://lists.freebsd.org/archives/freebsd-pf List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-pf@freebsd.org X-BeenThere: freebsd-pf@freebsd.org MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from mail.lacabanedeladmin.trickip.net (93.1.37.139) by MR2P264CA0039.FRAP264.PROD.OUTLOOK.COM (2603:10a6:500::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Thu, 9 Sep 2021 18:01:45 +0000 Received: from slackstro.home.lan ([172.16.93.19]) (authenticated bits=0) by mail.lacabanedeladmin.trickip.net (8.15.2/8.15.2) with ESMTPSA id 189I1WhR039955 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 9 Sep 2021 20:01:43 +0200 (CEST) (envelope-from kisscoolandthegangbang@hotmail.fr) X-MS-PublicTrafficType: Email X-IncomingHeaderCount: 45 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-Correlation-Id: d17f2b9e-006a-458a-f766-08d973bbe352 X-MS-TrafficTypeDiagnostic: VI1EUR06HT075: X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: BL4UUGI+2CIXjaZzf4AFheaX6Y/d5oKjtFxyld8hHVzgSpPApOMNREBOeMDIBeoQSwHGx3OIE6l4EJTiJa3PlI7SgucFYbOApmgie18Hwj30qbCqJjoJrJ2g8VfAfywmfik4L4SKVODWudr6s/oftVvnJe/ZnptnriocJVtS6/RPqQQIrnkHSZOs51fmRUSVg6iA/AEAiTeji/QruLBVyNJ31IpgCsDDt5HwZl+aHZa4WYXhhkEVum6l0xQjJam3Mw70b/3QHRF5N/9COLtBTGUFhlAFmTftM6u8SxEOVtcswZzEwQXFM6IvPong2jRRbOhc8k+ez5qXRhXgd0NMSpiSLIDkLacvOkLb6XyG40rAH4u/64NO6bJfEAATTrL301bxudifr1iuh3O4rlFQ1LjejBBYh/aHGg851CseyhJV39Cpn+j+i4wgaAqasHPG X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8qpjEt6/93akydSy/o5mNAXz4y4ZQzNH9ZuKhG7BfPxIuWI+6W8/AwR6PF1GDELL81SiRL6OiIn3FBV2VL907WFbMH4qYEC2B2qrGKowiTEcMWgAX6k8xrc4AFQzmAWr3ooN/vYvlEQeNssdLGHaNw== X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: d17f2b9e-006a-458a-f766-08d973bbe352 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Sep 2021 18:01:46.2377 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-AuthSource: VI1EUR06FT014.eop-eur06.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1EUR06HT075 X-Rspamd-Queue-Id: 4H56Jy3nJ0z4pQy X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=hotmail.fr; spf=pass (mx1.freebsd.org: domain of kisscoolandthegangbang@hotmail.fr designates 2a01:111:f400:fe11::830 as permitted sender) smtp.mailfrom=kisscoolandthegangbang@hotmail.fr X-Spamd-Result: default: False [-4.80 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; RCVD_COUNT_FIVE(0.00)[6]; RECEIVED_SPAMHAUS_PBL(0.00)[93.1.37.139:received]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[hotmail.fr]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_SPF_ALLOW(-0.20)[+ip6:2a01:111:f400::/48]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[hotmail.fr,none]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[hotmail.fr]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1] X-ThisMailContainsUnwantedMimeParts: N Hello, Cross posting because I am not sure where I am wrong here. I have a setup with some jails configured to use a dedicated virtual interface and with alternate routing tables/fibs. This is running on FreeBSD 11.4 amd64. The host has dual wan configuration. One adsl line via a router and one 4g via android phone. The problem I am facing is that I can use only one wan at a time depending on the combo routing/pf I use. I will go with the configuratin I would prefer to make it work. Here some config parts > # cat /etc/rc.local > ... > /sbin/ifconfig lo100 create >/dev/null > /sbin/ifconfig lo100 name vsw0 >/dev/null > /sbin/ifconfig vsw0 fib 1 >/dev/null > /sbin/route add default -iface vsw0 -fib 1 >/dev/null > ... One of my jail's config relevant part > ... > interface = "vsw0"; > ip4.addr += "vsw0|192.168.1.92/32"; > ... > exec.fib = 1; >... At the top of my pf.conf file, I have these lines > ... > no nat on $VSW from $proxout > nat on $phone_if tag PROXOUT tagged PROXOUTNAT -> ( $phone_if ) > nat on $lan_if tag PROXOUT tagged PROXOUTNAT -> $lan_ip > > pass out log quick on $VSW \ > proto tcp from $proxout to port {80, 443} user 100 tag PROXOUT100 no state > pass in log quick on $VSW tagged PROXOUT100 tag PROXOUTNAT rtable 0 > pass out log quick on $phone_if tagged PROXOUT rtable 0 > pass out log quick on $lan_if tagged PROXOUT rtable 0 > > block log quick from 109.0.64.169 > block log quick to 109.0.64.169 In this configuration, I can only use the secondary/backup (4G) line from the jail. This is done via routing some IPs via ue0 interface. When I delete these routes or change the routing to specifically use the adsl line (so same as default route), traffic is not coming back to the process but I can see something coming back to the host. > 19:45:46.210775 IP 192.168.1.50.57922 > 109.0.64.169.80: tcp 0 > 19:45:49.209728 IP 192.168.1.50.57922 > 109.0.64.169.80: tcp 0 > 19:45:49.216661 IP 109.0.64.169.80 > 192.168.1.50.57922: tcp 0 > 19:45:49.216816 IP 192.168.1.50.57922 > 109.0.64.169.80: tcp 0 > 19:45:49.217280 IP 192.168.1.50.57922 > 109.0.64.169.80: tcp 449 > 19:45:49.452641 IP 192.168.1.50.57922 > 109.0.64.169.80: tcp 449 > 19:45:49.716200 IP 192.168.1.50.57922 > 109.0.64.169.80: tcp 449 > 19:45:50.036820 IP 192.168.1.50.57922 > 109.0.64.169.80: tcp 449 > 19:45:50.216261 IP 109.0.64.169.80 > 192.168.1.50.57922: tcp 0 > 19:45:50.216355 IP 192.168.1.50.57922 > 109.0.64.169.80: tcp 0 > 19:45:50.476754 IP 192.168.1.50.57922 > 109.0.64.169.80: tcp 449 > 19:45:51.156785 IP 192.168.1.50.57922 > 109.0.64.169.80: tcp 449 > 19:45:52.216646 IP 109.0.64.169.80 > 192.168.1.50.57922: tcp 0 > 19:45:52.216725 IP 192.168.1.50.57922 > 109.0.64.169.80: tcp 0 > 19:45:52.316836 IP 192.168.1.50.57922 > 109.0.64.169.80: tcp 449 > 19:45:54.457517 IP 192.168.1.50.57922 > 109.0.64.169.80: tcp 449 > 19:45:55.964243 IP 192.168.1.50.57922 > 109.0.64.169.80: tcp 0 > 19:45:55.971939 IP 109.0.64.169.80 > 192.168.1.50.57922: tcp 0 > 19:45:58.498646 IP 192.168.1.50.57922 > 109.0.64.169.80: tcp 449 > 19:45:58.511970 IP 109.0.64.169.80 > 192.168.1.50.57922: tcp 1448 > 19:45:58.512087 IP 192.168.1.50.57922 > 109.0.64.169.80: tcp 0 > 19:45:58.512878 IP 109.0.64.169.80 > 192.168.1.50.57922: tcp 1168 > 19:45:58.512916 IP 192.168.1.50.57922 > 109.0.64.169.80: tcp 0 Log from pf: > 2021-09-09 19:45:46.210601 rule 0/0(match) [uid 100]: pass out on vsw0: > 192.168.1.92.13153 > 109.0.64.169.80: Flags [S], seq 3268104299, win 65535, > options [mss 16344,nop,wscale 6,sackOK,TS val 1426589561 ecr 0], length 0 > 2021-09-09 19:45:46.210670 rule 2/0(match): pass in on vsw0: > 192.168.1.92.13153 > 109.0.64.169.80: Flags [S], seq 3268104299, win 65535, > options [mss 16344,nop,wscale 6,sackOK,TS val 1426589561 ecr 0], length 0 > 2021-09-09 19:45:46.210746 rule 4/0(match): pass out on em0: > 192.168.1.50.57922 > 109.0.64.169.80: Flags [S], seq 3268104299, win 65535, > options [mss 16344,nop,wscale 6,sackOK,TS val 1426589561 ecr 0], length 0 In addition, traffic from my lan is ok and from host too. Both are using the adsl line. As I said I have another configuration example where this is the opposite. Traffic from adsl line is ok but from 4g timeouts. I don't think that's a problem with the jail as swapping configuration doesn't touch jail's conf. I don't really know where to look further. I try for some days now to make understand what happens. Maybe someone have an idea. Thanks, K.