From nobody Tue Jun 29 06:06:11 2021 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 1999D11F4577 for ; Tue, 29 Jun 2021 06:06:15 +0000 (UTC) (envelope-from gerrit.kuehn@aei.mpg.de) Received: from mailgate2.uni-hannover.de (mailgate2.uni-hannover.de [130.75.2.114]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GDYrQ2wQfz3rQd for ; Tue, 29 Jun 2021 06:06:14 +0000 (UTC) (envelope-from gerrit.kuehn@aei.mpg.de) Received: from intranet.aei.uni-hannover.de (ahin1.aei.uni-hannover.de [130.75.117.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mailgate2.uni-hannover.de (Postfix) with ESMTPS id 3DDD427D2 for ; Tue, 29 Jun 2021 08:06:12 +0200 (CEST) Received: from comet2.terra.ger ([87.146.77.63]) by intranet.aei.uni-hannover.de (IBM Domino Release 9.0.1FP8) with ESMTP id 2021062908061125-27094 ; Tue, 29 Jun 2021 08:06:11 +0200 Date: Tue, 29 Jun 2021 08:06:11 +0200 From: Gerrit Kuehn To: freebsd-net@freebsd.org Subject: Re: RFC: NFS trunking (multiple TCP connections for a mount Message-ID: <20210629080611.513b62d4@comet2.terra.ger> In-Reply-To: References: X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; amd64-portbld-freebsd12.1) List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 X-MIMETrack: Itemize by SMTP Server on intranet/aei-hannover(Release 9.0.1FP8|February 23, 2017) at 29/06/2021 08:06:11, Serialize by Router on intranet/aei-hannover(Release 9.0.1FP8|February 23, 2017) at 29/06/2021 08:06:11, Serialize complete at 29/06/2021 08:06:11 X-TNEFEvaluated: 1 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Virus-Scanned: clamav-milter 0.102.4 at mailgate2 X-Virus-Status: Clean X-Rspamd-Queue-Id: 4GDYrQ2wQfz3rQd X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of gerrit.kuehn@aei.mpg.de has no SPF policy when checking 130.75.2.114) smtp.mailfrom=gerrit.kuehn@aei.mpg.de X-Spamd-Result: default: False [-2.30 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-net@freebsd.org]; TO_DN_NONE(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; DMARC_NA(0.00)[mpg.de]; RCVD_IN_DNSWL_MED(-0.20)[130.75.2.114:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:680, ipnet:130.75.0.0/16, country:DE]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-net]; RECEIVED_SPAMHAUS_PBL(0.00)[87.146.77.63:received] X-ThisMailContainsUnwantedMimeParts: N On Tue, 29 Jun 2021 00:23:21 +0000 Rick Macklem wrote: > I don't understand how multiple TCP connections to the same > server IP address will distribute the load across multiple network > interfaces? > I thought that lagg would have handled this? I don't think this targets multiple interface situations. In my experience it is next to impossible to max. out a 10G connection with a single NFS connection running over it. The NICs have several independent queues like so (systat -ifstat): ixl0:aq ixl0:rxq0 ixl0:rxq1 ixl0:rxq2 ixl0:rxq3 ixl0:rxq4 ixl0:rxq5 ixl0:rxq6 ixl0:rxq7 A single connection will only use one of these, limiting the overall throughput. Spreading load across multiple connections sounds quite useful to me in this situation. cu Gerrit