From nobody Thu Dec 16 04:25:56 2021 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id DB11718EDF78; Thu, 16 Dec 2021 04:26:09 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4JDzZT3dFDz4prZ; Thu, 16 Dec 2021 04:26:09 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 1BG4PuvZ064486 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 16 Dec 2021 06:25:59 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 1BG4PuvZ064486 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 1BG4Pumq064485; Thu, 16 Dec 2021 06:25:56 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 16 Dec 2021 06:25:56 +0200 From: Konstantin Belousov To: Rick Macklem Cc: src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: 867c27c23a5c - main - nfscl: Change IO_APPEND writes to direct I/O Message-ID: References: <202112151639.1BFGdS2v011996@gitrepo.freebsd.org> List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <202112151639.1BFGdS2v011996@gitrepo.freebsd.org> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on tom.home X-Rspamd-Queue-Id: 4JDzZT3dFDz4prZ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Wed, Dec 15, 2021 at 04:39:28PM +0000, Rick Macklem wrote: > The branch main has been updated by rmacklem: > > URL: https://cgit.FreeBSD.org/src/commit/?id=867c27c23a5c469b27611cf53cc2390b5a193fa5 > > commit 867c27c23a5c469b27611cf53cc2390b5a193fa5 > Author: Rick Macklem > AuthorDate: 2021-12-15 16:35:48 +0000 > Commit: Rick Macklem > CommitDate: 2021-12-15 16:35:48 +0000 > > nfscl: Change IO_APPEND writes to direct I/O > > IO_APPEND writes have always been very slow over NFS, due to > the need to acquire an up to date file size after flushing > all writes to the NFS server. > > This patch switches the IO_APPEND writes to use direct I/O, > bypassing the buffer cache. As such, flushing of writes > normally only occurs when the open(..O_APPEND..) is done. > It does imply that all writes must be done synchronously > and must be committed to stable storage on the file server > (NFSWRITE_FILESYNC). > > For a simple test program that does 10,000 IO_APPEND writes > in a loop, performance improved significantly with this patch. > > For a UFS exported file system, the test ran 12x faster. > This drops to 3x faster when the open(2)/close(2) are done > for each loop iteration. > For a ZFS exported file system, the test ran 40% faster. > > The much smaller improvement may have been because the ZFS > file system I tested against does not have a ZIL log and > does have "sync" enabled. > > Note that IO_APPEND write performance is still much slower > than when done on local file systems. > > Although this is a simple patch, it does result in a > significant semantics change, so I have given it a > large MFC time. How is the buffer cache coherency is handled then? Imagine that other process either reads from this file, or even have it mapped. What does ensure that reads and page cache see the data written by direct path?