From nobody Sun Jun 16 21:18:33 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4W2QqF3x4nz5PVyF; Sun, 16 Jun 2024 21:18:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4W2QqF3Rxcz4GM1; Sun, 16 Jun 2024 21:18:33 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1718572713; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Na4NJg6lQePQALkN9f6mvmjSM2eLhftJQpSTfrfzQCQ=; b=U+n0iyQw8PirsNY4gRaHmt4oxkdoJAx7RucCtyq1OcaN46720j7SNXilKuA+3gSAmTWWSh /yuzw/kLznSdS5QVMhDNoGjIBCBJQnLu+K0Uny1VyhbUhvvKkUiNqzAoUYnyqn1MGIP2W9 e7cDhkTJgjKlKWn77s7zsQ5wn5vGEF2dYlEYSGd6vorbd2Bd96EtJrimbhhC2XieFQ9VJp zhDIkE+nuQhy8lYU+0bAahsJ1g5+A5516QP4x6BUDoogRZ6DEKeiRazfXBXyzGQ5Y2sy5b OLb9R85gHpwN34aHBzx0qMafBd/sd0oi7Fl9RGXJ+H6XjKicwKuprctNQUoVMg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1718572713; a=rsa-sha256; cv=none; b=MWZZBh/2DYmwcB45BAVLdQjzI2cCS3nTLV64diU41IdBnqFX/2CCO65SkDHe3ASW5Nf10F i8GXeLeWy88wA5SHFqsVd4OCh3+g9wloDUlqFkyHIzF9AgRx8ed50iLsI2lAqpBg+d50Uf uC5vA1EcG+U1o1WCX5EtxyxMi8AJzDPEtQ10AwJX8lj2awvA0nhMKz0j6DlKP7od7EOp14 mqOE/MmbWU5mauKzGU8YxJVQ6QFkg9smwZP6dvmxvsNWqaNkdViN9seCoDpCPtv2bLeq0q Tq6KdVcI6w2NMbk6iKIhXay+lz7ukCUhxyMCfGHlisElzjnncX4Ts75DD+gLOw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1718572713; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Na4NJg6lQePQALkN9f6mvmjSM2eLhftJQpSTfrfzQCQ=; b=vVlDcuMUBkGXrIkBCp3QLjgb3+mE5EoDwNZ8lnP4Q45UeIfbEF8O3g1MOc9SgXOFdy7LiY oRURN+nCT5+gCZzQx+tFpnYBntm4+ZwNp7d4up8iN3Qy9iMdxi7a4ZV7lhZOgxw01z6P+e ahELrpKONtEu2JG64F+xDSpVeYG6RImz+PeddyFtuu5k0cYFc4DNDP6dfXRnKpUUur6ukt Ytvskg24kQd6yIbclaXZJp1GZLk0CmLoqmtJjlcwFfC8jgnRDD2KALPxUxj8Rl1EcDkaTG 0aqnvEgX6oU4KL3AoI0Z1sbiDelTa5rPEehPftGBNxyXALyk/XF4hrx15L2iJQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4W2QqF2l99z13jt; Sun, 16 Jun 2024 21:18:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 45GLIXCA047796; Sun, 16 Jun 2024 21:18:33 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 45GLIXLP047793; Sun, 16 Jun 2024 21:18:33 GMT (envelope-from git) Date: Sun, 16 Jun 2024 21:18:33 GMT Message-Id: <202406162118.45GLIXLP047793@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Ryan Libby Subject: git: 0dc98b57f321 - main - getblk: track "non-sterile" bufobj to avoid bo lock on miss if sterile List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rlibby X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 0dc98b57f321945b24a160282eb402bf0a6e4220 Auto-Submitted: auto-generated The branch main has been updated by rlibby: URL: https://cgit.FreeBSD.org/src/commit/?id=0dc98b57f321945b24a160282eb402bf0a6e4220 commit 0dc98b57f321945b24a160282eb402bf0a6e4220 Author: Ryan Libby AuthorDate: 2024-06-16 21:09:45 +0000 Commit: Ryan Libby CommitDate: 2024-06-16 21:09:45 +0000 getblk: track "non-sterile" bufobj to avoid bo lock on miss if sterile This is a scheme to avoid taking the bufobj lock and doing a second lookup in the case where in getblk we do an unlocked lookup and find no buf. Was there really no buf, or were we in the middle of a reassignbuf race? By tracking any use of reassignbuf with a flag, we can know if there can't have been a race because there has been no reassignbuf. Because this scheme is spoiled on the first use of reassignbuf, it is mostly only beneficial for cases where a certain vnode is never expected to use dirty bufs at all. Reviewed by: kib Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D45571 --- sys/kern/vfs_bio.c | 8 +++++++- sys/kern/vfs_subr.c | 10 ++++++++++ sys/sys/bufobj.h | 1 + 3 files changed, 18 insertions(+), 1 deletion(-) diff --git a/sys/kern/vfs_bio.c b/sys/kern/vfs_bio.c index 4f1df9711dec..7bcc61c27109 100644 --- a/sys/kern/vfs_bio.c +++ b/sys/kern/vfs_bio.c @@ -4002,9 +4002,15 @@ getblkx(struct vnode *vp, daddr_t blkno, daddr_t dblkno, int size, int slpflag, /* * With GB_NOCREAT we must be sure about not finding the buffer * as it may have been reassigned during unlocked lookup. + * If BO_NONSTERILE is still unset, no reassign has occurred. */ - if ((flags & GB_NOCREAT) != 0) + if ((flags & GB_NOCREAT) != 0) { + /* Ensure bo_flag is loaded after gbincore_unlocked. */ + atomic_thread_fence_acq(); + if ((bo->bo_flag & BO_NONSTERILE) == 0) + return (EEXIST); goto loop; + } goto newbuf_unlocked; } diff --git a/sys/kern/vfs_subr.c b/sys/kern/vfs_subr.c index 398eda7ed897..f8f4c2a868f3 100644 --- a/sys/kern/vfs_subr.c +++ b/sys/kern/vfs_subr.c @@ -3207,6 +3207,16 @@ reassignbuf(struct buf *bp) bp, bp->b_vp, bp->b_flags); BO_LOCK(bo); + if ((bo->bo_flag & BO_NONSTERILE) == 0) { + /* + * Coordinate with getblk's unlocked lookup. Make + * BO_NONSTERILE visible before the first reassignbuf produces + * any side effect. This could be outside the bo lock if we + * used a separate atomic flag field. + */ + bo->bo_flag |= BO_NONSTERILE; + atomic_thread_fence_rel(); + } buf_vlist_remove(bp); /* diff --git a/sys/sys/bufobj.h b/sys/sys/bufobj.h index a6a9c5c2bf01..1c8fdcebe4d4 100644 --- a/sys/sys/bufobj.h +++ b/sys/sys/bufobj.h @@ -116,6 +116,7 @@ struct bufobj { #define BO_WWAIT (1 << 1) /* Wait for output to complete */ #define BO_DEAD (1 << 2) /* Dead; only with INVARIANTS */ #define BO_NOBUFS (1 << 3) /* No bufs allowed */ +#define BO_NONSTERILE (1 << 4) /* Ever called reassignbuf() */ #define BO_LOCKPTR(bo) (&(bo)->bo_lock) #define BO_LOCK(bo) rw_wlock(BO_LOCKPTR((bo)))