From nobody Sat Sep 16 23:01:42 2023 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Rp64l0gqfz4swr6 for ; Sat, 16 Sep 2023 23:01:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Rp64k6j11z3McT for ; Sat, 16 Sep 2023 23:01:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1694905302; a=rsa-sha256; cv=none; b=aKiEuIgmfX4RuFgHwUAs9fEjjk+9k6jkdB8Ay3QjdPt9ne/RW7ObUGghqD1tFBNLVpBMbK j+5imL4k+8aJsQylU4na7KUEWUK06oZWx3bpeXf7sub9NTHqTKX3J1JDQvidRs+Krk1RoQ 9Q8niXDBOdPsIWSjTRth13HwTtrPN21NFjROkuWk9vLcaYFG44UrbptuZh1KbNwioEc2h7 9t8KBbuS9sLxUSumZxvg2vY33TEmk8Kx/SJuz9dWlMAVn2gRIUHp22U518oxyqMqxtvcce 8ylqaAW1qJfR/WXwLCm5L3nZuzHNCOk1ZRHMXuoEQc+UlzGBGkwgEtKbok5tjA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1694905302; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VDqB3h+xTr/9YjQz3euWdB7uYSQTpwZC8DbbXnhUOa0=; b=f2iIcDwupYm+dSBmgN3xyj/uxKs3Gw8LwaSsTlZfC8EUZG84bskXLY3WyAmyyJs8J5zV10 zGtqmBQwnLLVGmzwhV5DvUBCl4LAbNwbm0Hw41u7bXlKSm1kSZWF4EKzfD1kwSRqFV54GO z0BVAKPDsG7H4+oTcgb6DsU7YbKipSArwo+/CYnm0qgWf4C1Q0lFps9mGPWzUK+p4GpsxJ WLhNUa3DzSMzmOyXIW2Zl3so3QPhNxuSu1zAKoztMsZ9+sFli0vmdoJmd6xj8j3tm33Mvz aZ4d0BthxFVyJeDoUlDGrQ5RzAbXi4kDuzXjTYzh++ZnAW+WqqkKXyYlLl4bIQ== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Rp64k5qHHz99G for ; Sat, 16 Sep 2023 23:01:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 38GN1gVw041277 for ; Sat, 16 Sep 2023 23:01:42 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 38GN1gVl041276 for fs@FreeBSD.org; Sat, 16 Sep 2023 23:01:42 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 267631] slow nfs from FreeBSD kvm guest/client to linux kvm host/server Date: Sat, 16 Sep 2023 23:01:42 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.0-RELEASE X-Bugzilla-Keywords: performance X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rmacklem@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D267631 Rick Macklem changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |rmacklem@FreeBSD.org --- Comment #6 from Rick Macklem --- As you've noted, NFS performance issues are often network interface related. However, here are a few tunables you can try, beyond "readahead", which was already mentioned. (And, yes, the man page is out of date w.r.t. readahead. You can build a kernel from sources with the value bumped up from 16, but I doubt a value greater than 16 will be needed?) First, do this on the client when the mount is established: # nfsstat -m This will show you what it is actually using. You probably have NFSv3,TCP and an rsize of 64K or 128K. If you stick "vfs.maxbcachebuf=3D1048576" in the client's /boot/loader.conf, the rsize will probably go up to 1Mbyte. (It will also recommend that you increase kern.ipc.maxsockbuf and will suggest a value. I'd increase it to at least the recommended value.) A large rsize/wsize will have a similar effect to increasing readahead, but will affect writing as well as reading. nocto - Close to Open consistency will improve correctness when multiple clients access the same files. If your files are not being manipulated by multiple clients concurrently, turning it off can help. Most 10Gbps net interfaces use multiple queues and pin a TCP connection to a queue. As such, an NFS mount with a single TCP connection can only get a fraction of the bandwidth. nconnect=3DN can help here, but it only works for NFSv4.1/4.2, so you also need to specify the "nfsv4" or "vers=3D4" mount option. (No idea w.r.t. VMs.) Then there are all the cache timeouts "acdirmin,...". Similar to nocto, longer timeouts have a negative impact if other clients (or processes locally on the server) are changing things. But longer timeouts result in better caching. Then there is "noatime", since few care about the access time being up to date. However, nothing above will fix poor performance caused by a poor net interface. --=20 You are receiving this mail because: You are the assignee for the bug.=