From nobody Thu Jun 09 04:15:17 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 7C157847215; Thu, 9 Jun 2022 04:15:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LJW392hnFz3QHL; Thu, 9 Jun 2022 04:15:17 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1654748117; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=X3lTyU3fSY1cdfb6QjGmoPWLmWdbKFqh1RNvZpoY198=; b=oZ3ka5v6mLf6kQAupizcquhR7uzZ9aEjPajuNn2bGVPNGlfB4POa32XzoHwI8skOGSpphM 4pbKBCqeHXxcvfz0IN3L/eK8VXFFPglohPh9Oz+Z56wWIQYqpzq9ONuZx1XcQOZHSbEc9Y E0TpvEUOL0v1js9xYDDEU2Izlp+O66K2t6sNyraq8xHrwDH7y1lKV+5LFm6tq1oGLmffUL ka1s1E3httOknp+TqMHnv3Tg3+eTkelEr8NSktk2EgHm6yM8axd8cL/uUj0KAwi80QPKxo QmkujQGB347LfJ0yhUh2zYlrAOLiXsJ61vSUdp+zSy73mTh5+8igDqw22HULLw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 39AEA1D47A; Thu, 9 Jun 2022 04:15:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 2594FHk6096083; Thu, 9 Jun 2022 04:15:17 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 2594FHUc096082; Thu, 9 Jun 2022 04:15:17 GMT (envelope-from git) Date: Thu, 9 Jun 2022 04:15:17 GMT Message-Id: <202206090415.2594FHUc096082@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Rick Macklem Subject: git: 703bbad7f459 - stable/13 - nfscl: Change IO_APPEND writes to direct I/O List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 703bbad7f459a0f9c09c2dd470ba5bfc669b7c57 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1654748117; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=X3lTyU3fSY1cdfb6QjGmoPWLmWdbKFqh1RNvZpoY198=; b=GWZX+CdoYnFULejIEpytZoFjZbLnwjeOF+VnkLrWyfFNCMJuk/H002ts2dap85ORjmMiVC DgyjWunKeAV2C08GHM3kSlBELDK42kATgaI1caWW2ayzSrBZwBdgl9yHSc/O+0z6G9klAz Bjp4ghDV4LEi/iswAqjp7JQ3XQLCCY1JwshZyzbyY3YOg+633WbQ3QvjHvCtoAKGvExnwc QubZH8lQ54B/KJaIOfnacgJ8jj8Hdtyo19wVjTXrJ3KmN8FUB2dh7uftjSNf0bEIBW/BmQ HZFTTR0PwHRRTaal+lyZgWBAhPJ0JjQ2n2gf6V3ZJfuL/RZTeG0D1LcXFbN/cw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1654748117; a=rsa-sha256; cv=none; b=oWQ9p/fN80T6gzb9ctCO4ZpJT3UbRauQWBzmgogltG9ioM8+T6ccKn3F1DG6MgKdeG20I/ KHMoYkmSM9kCjfId/c9OgVuSI3S+r5mREpE9qe4PlwfkKuI9BF+UkrU1W++tnMFsdjOhbV HnW8Z+E+fXNqWTkx0OWJ1+QvAI0MP5DuhKQAZMj2CYfbZ3+vgQJPDGME3rr0PPyRY6ovN3 Vk6vyc4f5OoZM15UuQwsQhtwVLjEWSwQPnTioufJEVTaSjkBR0WUFWOlXFnJduzPLk/0wk 8TIUmgvCeGiZl7L+C7u6jKPYO1hi8nQSR567Dj29lBzgjc7x9NQ1lmZJa5yJzg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=703bbad7f459a0f9c09c2dd470ba5bfc669b7c57 commit 703bbad7f459a0f9c09c2dd470ba5bfc669b7c57 Author: Rick Macklem AuthorDate: 2021-12-15 16:35:48 +0000 Commit: Rick Macklem CommitDate: 2022-06-09 04:14:21 +0000 nfscl: Change IO_APPEND writes to direct I/O IO_APPEND writes have always been very slow over NFS, due to the need to acquire an up to date file size after flushing all writes to the NFS server. This patch switches the IO_APPEND writes to use direct I/O, bypassing the buffer cache. As such, flushing of writes normally only occurs when the open(..O_APPEND..) is done. It does imply that all writes must be done synchronously and must be committed to stable storage on the file server (NFSWRITE_FILESYNC). For a simple test program that does 10,000 IO_APPEND writes in a loop, performance improved significantly with this patch. For a UFS exported file system, the test ran 12x faster. This drops to 3x faster when the open(2)/close(2) are done for each loop iteration. For a ZFS exported file system, the test ran 40% faster. The much smaller improvement may have been because the ZFS file system I tested against does not have a ZIL log and does have "sync" enabled. Note that IO_APPEND write performance is still much slower than when done on local file systems. Although this is a simple patch, it does result in a significant semantics change, so I have given it a large MFC time. (cherry picked from commit 867c27c23a5c469b27611cf53cc2390b5a193fa5) --- sys/fs/nfsclient/nfs_clbio.c | 6 +++++- sys/fs/nfsclient/nfs_clvnops.c | 8 ++++---- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/sys/fs/nfsclient/nfs_clbio.c b/sys/fs/nfsclient/nfs_clbio.c index 6eee88fce9d3..908f59855730 100644 --- a/sys/fs/nfsclient/nfs_clbio.c +++ b/sys/fs/nfsclient/nfs_clbio.c @@ -998,8 +998,12 @@ ncl_write(struct vop_write_args *ap) if (uio->uio_resid == 0) return (0); - if (newnfs_directio_enable && (ioflag & IO_DIRECT) && vp->v_type == VREG) + if (vp->v_type == VREG && ((newnfs_directio_enable && (ioflag & + IO_DIRECT)) || (ioflag & IO_APPEND))) { + if ((ioflag & IO_APPEND) != 0) + ioflag |= IO_SYNC; return nfs_directio_write(vp, uio, cred, ioflag); + } /* * Maybe this should be above the vnode op call, but so long as diff --git a/sys/fs/nfsclient/nfs_clvnops.c b/sys/fs/nfsclient/nfs_clvnops.c index 38b637c5c062..538f029a8bfd 100644 --- a/sys/fs/nfsclient/nfs_clvnops.c +++ b/sys/fs/nfsclient/nfs_clvnops.c @@ -707,8 +707,8 @@ nfs_open(struct vop_open_args *ap) /* * If the object has >= 1 O_DIRECT active opens, we disable caching. */ - if (newnfs_directio_enable && (fmode & O_DIRECT) && - (vp->v_type == VREG)) { + if (vp->v_type == VREG && ((newnfs_directio_enable && (fmode & + O_DIRECT)) || (fmode & O_APPEND))) { if (np->n_directio_opens == 0) { NFSUNLOCKNODE(np); if (VOP_ISLOCKED(vp) != LK_EXCLUSIVE) { @@ -965,11 +965,11 @@ nfs_close(struct vop_close_args *ap) error = nfscl_maperr(ap->a_td, error, (uid_t)0, (gid_t)0); } - if (newnfs_directio_enable) + if (vp->v_type == VREG && ((newnfs_directio_enable && (fmode & + O_DIRECT)) || (fmode & O_APPEND))) { KASSERT((np->n_directio_asyncwr == 0), ("nfs_close: dirty unflushed (%d) directio buffers\n", np->n_directio_asyncwr)); - if (newnfs_directio_enable && (fmode & O_DIRECT) && (vp->v_type == VREG)) { NFSLOCKNODE(np); KASSERT((np->n_directio_opens > 0), ("nfs_close: unexpectedly value (0) of n_directio_opens\n"));