From nobody Mon Oct 30 13:47:26 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SJvk12txGz4xxQC for ; Mon, 30 Oct 2023 13:48:25 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SJvk01Nm8z3CW4 for ; Mon, 30 Oct 2023 13:48:24 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=AjKTfDy8; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=lAjLhCM2; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.27 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id BC9755C01A5 for ; Mon, 30 Oct 2023 09:48:23 -0400 (EDT) Received: from imap46 ([10.202.2.96]) by compute6.internal (MEProxy); Mon, 30 Oct 2023 09:48:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1698673703; x=1698760103; bh=t/ C3TpCCd+XlXcI673X4/NK3AWuZ2u9jDKkOcDvehbA=; b=AjKTfDy8EY7S46WzzV ulXoNPUZUXt51kn5h3b4IbohO8ZSqhFLFq5XX1mOrlKSUWtr0Zkxw02GYGHErR12 WNcnzDFxyWXr/a3fWin/kXROhSfHPTYyb0zzMdHvjJQLgkScPGGf9+3E4lU6nUc/ z/l7ZzkRm2LzTo5ygTdugKe1mxlY7yay9T0eWNR/p6jVlSSTxTDafgGqtnZxjJsr /A3BauxOcSbcnZLtLP9dSuwko7ES3YF70vG0IYox0oXf23YVfkA2BzMCkfqrzxnq SjQxIzpthw9y4RFi3ISGvwQ4U/i0aY+RjnA+MYmOGbvjiz5mznz/rc4eCxZf74JT 81GA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1698673703; x=1698760103; bh=t/C3TpCCd+XlX cI673X4/NK3AWuZ2u9jDKkOcDvehbA=; b=lAjLhCM2xz0vVE+Idt7mfcOU87gl+ yah8Y2Ju9gdBnVAcMYIaFnIAClHTY2cAm5pAdA/kef+E5RMv3m4r872HBMuf0sAT gRfVSpQcuNpoRpQ2yBrGioV3/bvnYv/1O0+F4bbP84jiYcA944wQM3ZeOT6vSEBm xqRDl1YynGuli0w8BQj8TzkdoNpSDIMu4qK3/T7WLg3dWF2hhol22xHk+Z8aUiNq 2PDfcSY8yTCTgTwKrSY2Nb9Sv2Dj9vAQSQ6t8Kc/z1DymcMrkwopGzoWzJTCi2TJ dsv9F6hIahOibPw8/9rA6iwPQgf+8mZMRp8MYV9c9s1NcFk3feoVrNwQA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedruddttddgheegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderredtnecuhfhrohhmpehvohhiugcuoehvohhiugesfhdqmhdrfhhmqeenucggtffr rghtthgvrhhnpeeitedvueehtdehtddvhfeuhfevhedvieelvdeiffehveelheegfedule ejudekvdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhm pehvohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id 825942A20085; Mon, 30 Oct 2023 09:48:23 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.9.0-alpha0-1048-g9229b632c5-fm-20231019.001-g9229b632 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Message-Id: <71fc15b3-521a-4701-9c53-f06570d4b97a@app.fastmail.com> In-Reply-To: References: Date: Mon, 30 Oct 2023 13:47:26 +0000 From: void To: freebsd-fs Subject: Re: optimising nfs and nfsd Content-Type: text/plain X-Spamd-Result: default: False [-5.03 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.994]; NEURAL_HAM_MEDIUM(-0.95)[-0.948]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.27]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.27:from]; XM_UA_NO_VERSION(0.01)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[] X-Rspamd-Queue-Id: 4SJvk01Nm8z3CW4 X-Spamd-Bar: ----- Hi Rick, thanks for the info On Sun, 29 Oct 2023, at 20:28, Rick Macklem wrote: > In summary, if you are getting near wire speed and you > are comfortable with your security situation, then there > isn't much else to do. It seems to depend on the nature of the workload. Sometimes wire speed, sometimes half that. And then: 1. some clients - many reads of small files, hardly any writes 2. others - many reads, loads of writes 3. same as {1,2} above, huge files 4. how many clients access at once 5. how many clients of [1] and [2] types access at the same time looking for an all-in-one synthetic tester if there's such a thing. Large single client transfers client to server are wire speed. Not tested much else, (not sure how), except with dd but that's not really a real-world workload. I'll try the things you suggested. what I can report now, on the server, so before nfs is considered: dd if=/dev/urandom of=test-128k.bin bs=128k count=64000 status=progress 8346009600 bytes (8346 MB, 7959 MiB) transferred 59.001s, 141 MB/s dd if=test-128k.bin of=/dev/null bs=128k status=progress 6550061056 bytes (6550 MB, 6247 MiB) transferred 3.007s, 2178 MB/s dd if=/dev/urandom of=test-4k.bin bs=4k count=2048000 status=progress 8301215744 bytes (8301 MB, 7917 MiB) transferred 78.063s, 106 MB/s dd if=test-4k.bin of=/dev/null bs=4k status=progress 7725998080 bytes (7726 MB, 7368 MiB) transferred 10.002s, 772 MB/s dd if=/dev/urandom of=test-512b.bin bs=512 count=16384000 status=progress 8382560256 bytes (8383 MB, 7994 MiB) transferred 208.019s, 40 MB/s dd if=test-512b.bin of=/dev/null bs=512 status=progress 8304610304 bytes (8305 MB, 7920 MiB) transferred 63.062s, 132 MB/s