From nobody Fri Dec 27 08:48:48 2024 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YKK0j3t3gz5jR7R for ; Fri, 27 Dec 2024 08:48:53 +0000 (UTC) (envelope-from paul@redbarn.org) Received: from util.redbarn.org (util.redbarn.org [IPv6:2001:559:8000:cd::222]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "*.redbarn.org", Issuer "RapidSSL TLS RSA CA G1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YKK0h2qp2z59Qr for ; Fri, 27 Dec 2024 08:48:52 +0000 (UTC) (envelope-from paul@redbarn.org) Authentication-Results: mx1.freebsd.org; dkim=fail ("headers rsa verify failed") header.d=redbarn.org header.s=util header.b=KFjDFEQC; spf=pass (mx1.freebsd.org: domain of paul@redbarn.org designates 2001:559:8000:cd::222 as permitted sender) smtp.mailfrom=paul@redbarn.org; dmarc=pass (policy=reject) header.from=redbarn.org Received: from family.redbarn.org (family.redbarn.org [24.104.150.213]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "*.redbarn.org", Issuer "RapidSSL TLS RSA CA G1" (not verified)) by util.redbarn.org (Postfix) with ESMTPS id 7E132160C24; Fri, 27 Dec 2024 08:48:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=redbarn.org; s=util; t=1735289328; bh=NWklz6Cs88bWG2bop3LaEET1E0Mru8KTEsUhcHPPxgA=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=KFjDFEQC/xbo4a1BXeE3op/O8yuR9XEHcId7icvZ6rpBXNv2IvlpxnIJuaQs/lgxc axH5qeaLatHQBZT40I9D89rKj3lhPtyb9uFqXzPQ7P/qHQsoWcpOVR9ftP8Irlq7WL k2jpbT5KUk/ps+O1mkMJkdHUlCpSCaae0TyEhL4M= Received: from dhcp-151.access.rits.tisf.net (dhcp-151.access.rits.tisf.net [24.104.150.151]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by family.redbarn.org (Postfix) with ESMTPSA id 4F179C3F22; Fri, 27 Dec 2024 08:48:48 +0000 (UTC) From: Paul Vixie To: Santiago Martinez , Jamie Landeg-Jones Cc: freebsd-net@freebsd.org Subject: Re: per-FIB socket binding Date: Fri, 27 Dec 2024 08:48:48 +0000 Message-ID: <38589000.XM6RcZxFsP@dhcp-151.access.rits.tisf.net> Organization: FW In-Reply-To: <28EF197D-0D10-449A-A3C5-8B931F31CA6C@codenetworks.net> References: <7772475.EvYhyI6sBW@dhcp-151.access.rits.tisf.net> <28EF197D-0D10-449A-A3C5-8B931F31CA6C@codenetworks.net> List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="nextPart2734174.Isy0gbHreE" Content-Transfer-Encoding: 7Bit X-Spamd-Result: default: False [-3.38 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; NEURAL_HAM_MEDIUM(-0.78)[-0.778]; CTE_CASE(0.50)[]; DMARC_POLICY_ALLOW_WITH_FAILURES(-0.50)[]; RCVD_IN_DNSWL_MED(-0.20)[24.104.150.213:received]; R_SPF_ALLOW(-0.20)[+ip6:2001:559:8000::/48]; MIME_GOOD(-0.10)[multipart/mixed,multipart/alternative,text/plain,text/x-patch]; RCVD_IN_DNSWL_LOW(-0.10)[2001:559:8000:cd::222:from]; RCVD_TLS_ALL(0.00)[]; DMARC_POLICY_ALLOW(0.00)[redbarn.org,reject]; FROM_HAS_DN(0.00)[]; HAS_ATTACHMENT(0.00)[]; R_DKIM_REJECT(0.00)[redbarn.org:s=util]; HAS_ORG_HEADER(0.00)[]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[redbarn.org:-]; FREEFALL_USER(0.00)[paul]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; MLMMJ_DEST(0.00)[freebsd-net@freebsd.org]; MISSING_XM_UA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:33651, ipnet:2001:559:8000::/48, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~,4:+] X-Rspamd-Queue-Id: 4YKK0h2qp2z59Qr X-Spamd-Bar: --- This is a multi-part message in MIME format. --nextPart2734174.Isy0gbHreE Content-Type: multipart/alternative; boundary="nextPart1970228.IobQ9Gjlxr" Content-Transfer-Encoding: 7Bit This is a multi-part message in MIME format. --nextPart1970228.IobQ9Gjlxr Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" On Tuesday, December 24, 2024 3:34:45 AM UTC Santiago Martinez wrote: > Hi, > here=E2=80=99s another user of fibs. Each of our servers have multiple fi= bs and > jails with fibs. I like the proposed. > Santi Cool. Read on. On Tuesday, December 24, 2024 5:06:32 AM UTC Jamie Landeg-Jones wrote: > Paul Vixie wrote: > > ... > I like that. I isolate 5 seperate networks by assigning a fib to each > interface, and was initially surprised that I had to jump through ipfw > hoops to get it to work properly, in fact at the end of my ipfw rules for > these interfaces, just to guarantee no leaking, ... >=20 > So, yes, I agree that it's crocky, and your proposal is how I originally > expected it to work, and indeed, I can so no reason for it not to work th= at > way, but am prepared to be enlightened if anyone else has an opinion on > this. >=20 > Jamie Groovy. See attached patch. This is just for TCP since I have no way to tes= t SCTP and I=20 think UDP will have to be handled at the application layer. There are two o= ne line changes=20 here. =46irst, save the FIB number from the SYN in the syncache. This FIB number = was in the=20 incoming m_pkthdr so I didn't need to change any function signatures. Note = that if the=20 listener socket has a non-zero FIB number it will be used instead of the in= terface FIB=20 number -- it's more specific and likely to be right. Second, when the initial ACK arrives and it's time for the connection to ex= it from the=20 syncache and to become a socket, restore the original FIB number and apply = it to the=20 cloned socket, which will already have inherited its FIB number from the li= stener socket. This works here. The diff is for a 14.2 kernel but is likely backward-porta= ble. I'd very much=20 like to hear anybody's experience with this patch, or commentary on its app= roach and/or=20 advisability. =2D-=20 Paul Vixie --nextPart1970228.IobQ9Gjlxr Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="UTF-8"

On Tuesday, December 24, 2024 3:34:45 AM UTC Santiago Martinez wrote:

>= ; Hi,

>= ; here=E2=80=99s another user of fibs. Each of our servers have multiple fi= bs and

>= ; jails with fibs. I like the proposed.

>= ; Santi


Cool. Read on.


On Tuesday, December 24, 2024 5:06:32 AM UTC Jamie Landeg-Jones wrote:

>= ; Paul Vixie <paul@redbarn.org> wrote:

>= ; > ...

>= ; I like that. I isolate 5 seperate networks by assigning a fib to each

>= ; interface, and was initially surprised that I had to jump through ipfw

>= ; hoops to get it to work properly, in fact at the end of my ipfw rules for=

>= ; these interfaces, just to guarantee no leaking, ...

>= ;

>= ; So, yes, I agree that it's crocky, and your proposal is how I originally<= /p>

>= ; expected it to work, and indeed, I can so no reason for it not to work th= at

>= ; way, but am prepared to be enlightened if anyone else has an opinion on

>= ; this.

>= ;

>= ; Jamie


Groovy. See attached patch. This is just for TCP since I have no way to = test SCTP and I think UDP will have to be handled at the application layer.= There are two one line changes here.


First, save the FIB number from the SYN in the syncache. This FIB number= was in the incoming m_pkthdr so I didn't need to change any function signa= tures. Note that if the listener socket has a non-zero FIB number it will b= e used instead of the interface FIB number -- it's more specific and likely= to be right.


Second, when the initial ACK arrives and it's time for the connection to= exit from the syncache and to become a socket, restore the original FIB nu= mber and apply it to the cloned socket, which will already have inherited i= ts FIB number from the listener socket.


This works here. The diff is for a 14.2 kernel but is likely backward-po= rtable. I'd very much like to hear anybody's experience with this patch, or= commentary on its approach and/or advisability.


--

Pau= l Vixie

--nextPart1970228.IobQ9Gjlxr-- --nextPart2734174.Isy0gbHreE Content-Disposition: attachment; filename="fibnum.diff" Content-Transfer-Encoding: 7Bit Content-Type: text/x-patch; charset="x-UTF_8J"; name="fibnum.diff" diff --git a/sys/netinet/tcp_input.c b/sys/netinet/tcp_input.c index 83f85a50e..0e030f24f 100644 --- a/sys/netinet/tcp_input.c +++ b/sys/netinet/tcp_input.c @@ -1057,7 +1057,7 @@ tcp_input_with_port(struct mbuf **mp, int *offp, int proto, uint16_t port) } inc.inc_fport = th->th_sport; inc.inc_lport = th->th_dport; - inc.inc_fibnum = so->so_fibnum; + inc.inc_fibnum = so->so_fibnum || m->m_pkthdr.fibnum; /* * Check for an existing connection attempt in syncache if diff --git a/sys/netinet/tcp_syncache.c b/sys/netinet/tcp_syncache.c index 15244a61d..a50648fa5 100644 --- a/sys/netinet/tcp_syncache.c +++ b/sys/netinet/tcp_syncache.c @@ -805,6 +805,7 @@ syncache_socket(struct syncache *sc, struct socket *lso, struct mbuf *m) */ if ((so = solisten_clone(lso)) == NULL) goto allocfail; + so->so_fibnum = sc->sc_inc.inc_fibnum; #ifdef MAC mac_socketpeer_set_from_mbuf(m, so); #endif --nextPart2734174.Isy0gbHreE--