From nobody Sat Jan 25 05:15:58 2025 X-Original-To: freebsd-geom@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Yg2vz4b2Hz5m0qj for ; Sat, 25 Jan 2025 05:16:15 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Yg2vy2sZXz3dxS for ; Sat, 25 Jan 2025 05:16:14 +0000 (UTC) (envelope-from kostikbel@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) Received: from tom.home (kib@localhost [127.0.0.1] (may be forged)) by kib.kiev.ua (8.18.1/8.18.1) with ESMTP id 50P5FwmO090098; Sat, 25 Jan 2025 07:16:01 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 50P5FwmO090098 Received: (from kostik@localhost) by tom.home (8.18.1/8.18.1/Submit) id 50P5FwuW090097; Sat, 25 Jan 2025 07:15:58 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 25 Jan 2025 07:15:58 +0200 From: Konstantin Belousov To: Jonathan Krebs Cc: freebsd-geom@freebsd.org Subject: Re: when does BIO_FLUSH happen on a disk? Message-ID: References: List-Id: GEOM-specific discussions and implementations List-Archive: https://lists.freebsd.org/archives/freebsd-geom List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-geom@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=4.0.1 X-Spam-Checker-Version: SpamAssassin 4.0.1 (2024-03-26) on tom.home X-Spamd-Result: default: False [0.23 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_SPAM_MEDIUM(0.99)[0.987]; NEURAL_HAM_SHORT(-0.96)[-0.962]; NEURAL_HAM_LONG(-0.80)[-0.795]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none]; MIME_GOOD(-0.10)[text/plain]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; MISSING_XM_UA(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ARC_NA(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[freebsd-geom@freebsd.org]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_DKIM_NA(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all:c]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_SOME(0.00)[]; HAS_XAW(0.00)[] X-Spamd-Bar: / X-Rspamd-Queue-Id: 4Yg2vy2sZXz3dxS On Fri, Jan 24, 2025 at 08:10:01PM +0100, Jonathan Krebs wrote: > Dear FreeBSD developers, > > Inside a QEMU-VM, I inserted calls to g_print_bio in the first lines of > sys/cam/ata/ata_da.c:adastrategy() and > sys/dev/virtio/block/virtio_blk.c:vtblk_strategy() of a FreeBSD 14.2, and > added an image file with -drive file=test.img,format=raw[,if=virtio]. > > While using the virtual disk, I observe BIO_WRITE request, but never a > BIO_FLUSH. > Usage was by > - creating a UFS filesystem (newfs /dev/vtbd0) > - mounting it in /mnt > - write a file with > dd of=/mnt/a bs=4k count=20 if=/dev/urandom > sync > dd of=/mnt/a bs=4k count=20 if=/dev/urandom conv=fdatasync > dd of=/mnt/a bs=4k count=20 if=/dev/urandom oflag=sync The only case where UFS issues BIO_FLUSH, together with BIO_ORDERED flag, to prevent reordering of the requests before/after the flush, is when a journal segment is written. This is done to ensure that metadata blocks are written after the conduit journal records are on the stable storage. Generally, UFS uses softupdates technique and relies on the callbacks from the buffer write completions to update internal structures to indicate metadata write progress. With such approach, there is no need to issue flushes. For instance, when doing fsync(2) and scheduling writes required to flush the metadata and data, we only return from syscall when disk reported that all writes were done. The case above (for the journal segment write) could be considered as a way to short-cut some additional code that would otherwise need to be written. > > - repeating this with a geli layer (no special parameters to geli init) and > with msdosfs instead of UFS > > The differences between the dd calls with and without sync options was that > the WRITEs were immediately issued with conv=fdatasync or oflag=sync, but no > FLUSHs were observed (with one exception: geli clear) > > This is why I wonder when BIO_FLUSH is used and if it means something > different. > > I am interested for these reasons: > > - In a university research project, I am trying to suspend+resume a system > without calling the DEVICE_SUSPEND functions. Therefore I wonder when I can > assume that a BIO_WRITE has really arrived on disk > > - The GELI class seems to reorder WRITEs and FLUSHes. (at least in my > understanding while looking for code example on writing a geom class) This > was also discussed here 12 years ago, where I could not find a resolution: https://freebsd-geom.freebsd.narkive.com/KruBpS7r/geli-and-bio-flush-and-or-bio-ordered-issue > (writes are queued, flush is passed g_down immediately, > https://cgit.freebsd.org/src/tree/sys/geom/eli/g_eli.c#n507 ) > > Thanks and Happy Hacking, > thejonny