From nobody Tue Jun 29 06:28:41 2021
X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
	by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 093FA11F5C7A
	for <freebsd-net@mlmmj.nyi.freebsd.org>; Tue, 29 Jun 2021 06:29:01 +0000 (UTC)
	(envelope-from lutz@iks-jena.de)
Received: from annwfn.iks-jena.de (annwfn.iks-jena.de [IPv6:2001:4bd8::19])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client did not present a certificate)
	by mx1.freebsd.org (Postfix) with ESMTPS id 4GDZLh5CCSz3sql
	for <freebsd-net@freebsd.org>; Tue, 29 Jun 2021 06:29:00 +0000 (UTC)
	(envelope-from lutz@iks-jena.de)
X-SMTP-Sender: IPv6:2001:4bd8:0:666:248:54ff:fe12:ee3f
Received: from belenus.iks-jena.de (belenus.iks-jena.de [IPv6:2001:4bd8:0:666:248:54ff:fe12:ee3f])
	by annwfn.iks-jena.de (8.15.2/8.15.2) with ESMTPS id 15T6SgMg030390
	(version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT);
	Tue, 29 Jun 2021 08:28:42 +0200
X-MSA-Host: belenus.iks-jena.de
Received: (from lutz@localhost)
	by belenus.iks-jena.de (8.14.3/8.14.1/Submit) id 15T6Sf4X031665;
	Tue, 29 Jun 2021 08:28:41 +0200
Date: Tue, 29 Jun 2021 08:28:41 +0200
From: Lutz Donnerhacke <lutz@donnerhacke.de>
To: Rick Macklem <rmacklem@uoguelph.ca>
Cc: freebsd-net <freebsd-net@freebsd.org>
Subject: Re: RFC: NFS trunking (multiple TCP connections for a mount
Message-ID: <20210629062841.GC31225@belenus.iks-jena.de>
References: <YQXPR0101MB0968DC173855A82AAF45F08FDD039@YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM>
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Archive: https://lists.freebsd.org/archives/freebsd-net
List-Help: <mailto:freebsd-net+help@freebsd.org>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Subscribe: <mailto:freebsd-net+subscribe@freebsd.org>
List-Unsubscribe: <mailto:freebsd-net+unsubscribe@freebsd.org>
Sender: owner-freebsd-net@freebsd.org
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <YQXPR0101MB0968DC173855A82AAF45F08FDD039@YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM>
X-message-flag: Please send plain text messages only. Thank you.
User-Agent: Mutt/1.5.17 (2007-11-01)
X-Rspamd-Queue-Id: 4GDZLh5CCSz3sql
X-Spamd-Bar: ----
Authentication-Results: mx1.freebsd.org;
	none
X-Spamd-Result: default: False [-4.00 / 15.00];
	 REPLY(-4.00)[]
X-ThisMailContainsUnwantedMimeParts: N

On Tue, Jun 29, 2021 at 12:23:21AM +0000, Rick Macklem wrote:
> I don't understand how multiple TCP connections to the same
> server IP address will distribute the load across multiple network
> interfaces?

Sounds like a bandwith-latency-product issue. TCP is prone to stalling at
the end of a buffer window (in addition to reassembly and RPC waits).
Multiple TCP streams scale linear with the available bandwidth over them.
Single RPC messages should not split over multiple streams (causes reorder
of segments), but different messages can go in round robin fashion. There is
no need to ensure a symmetric path, so that the response comes back on the
same stream.