From nobody Wed Sep 06 21:56:39 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Rgx6J008gz4sJ75; Wed, 6 Sep 2023 21:56:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Rgx6H5HMGz3bCQ; Wed, 6 Sep 2023 21:56:39 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1694037399; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Y6+PZGbGD++BDT922AoyeHp0YovyKvc3rUnAchQmKac=; b=qBB1jghfSELdi3zYVV4eix05uP2aRW+OYTb7W2/H4KpH4IDPKv6OhMU3u8cq7ZN12tRXiz kwvZMlr0SAC2sbSSQg7sT7/8F1QE5ULDaCxOu59kweV19odL8Aeiqsj6XVf1u2HLesxgN1 LOVvsArOVYJDCy8UoI5CgoyfJfJIrJDG7axKM9GYM3B2WZwIwsiXVTSYUJLV5UJ1s4w6Ao Yzil4hqp7UgVu48h2OBs3WDyBGpoABWLRIaYTkSOpfkHsX4uC1BFyfv0xZYdQfj1mALn3J Ugb8oG8+XAUhiTPl/LVvXkJkAasjDGyz31rKOyJb1KGq+gmuk22W+Da+0SUoyg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1694037399; a=rsa-sha256; cv=none; b=SDqzJgEWtrGp6JQGtK3kYP2ofIuGGyQGtrOyymLI/NC7ex+EMh8KD/tYfU3gxx5arA/+Me sdkfBY8+klfQ+DZhhcdhBNvibWMlkwRG60WX7eiXwsjSQ2Uj2zTAjFc57YHAtnqaEZCJav kJOA4YPk3Txc4BUcxV84d7F1EIVbRDQf7mWAAjot/4EppF+yIzh5TIA+/FEQoNcTAoXvhY sjhBeG2wLr726+O0Lolc26xBKkuK/uC3VnRstMKeJia1sHoE9HLTfe7qxGmJXg1zkPdqJa GtNYWW3s5E9LfL5iq78oYly6KNefFQB7qYoCgNza3DluhHyI5ChdlaOLXk2J6Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1694037399; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Y6+PZGbGD++BDT922AoyeHp0YovyKvc3rUnAchQmKac=; b=wOcYDZou1Fm9/fdfOnRSEcFBCDEjKlBRDg79MCYDMBs3BnQ3EV8J6csvQzmeJaBWyY+Mbw Nv1NovvEG36s2UCpEknUPBwkdBC2O+ne7G5xNWXL22Kzp8ou1PzjcVMSKJiLlRnf9mkrZq OsLAQ4BxIpQ2/CWdTQc6BDQNXC9UxDAbkzBT9Y3z6uDd1ZYS8vOws/2eF+oAOU8quVEaLK bTr8ZUZV1V5T5TtViamoG0HOhk3o7+zXQImETMDbEighInTrzfEB2+1NGbSepcR5QCUNLN 6QpyTCWvtvw2+Nhc0Er4Os3eTEkp2aJoE9biMpFMwDzDnejxLEKIjHYRdTEe2w== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Rgx6H4PDYzZ6D; Wed, 6 Sep 2023 21:56:39 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 386LudpU022715; Wed, 6 Sep 2023 21:56:39 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 386Ludqj022712; Wed, 6 Sep 2023 21:56:39 GMT (envelope-from git) Date: Wed, 6 Sep 2023 21:56:39 GMT Message-Id: <202309062156.386Ludqj022712@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: John Baldwin Subject: git: fe6953f70fab - stable/13 - aio: Fix more synchronization issues in aio_biowakeup. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: fe6953f70fab1f89de198d3dbc885d6f89a44883 Auto-Submitted: auto-generated The branch stable/13 has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=fe6953f70fab1f89de198d3dbc885d6f89a44883 commit fe6953f70fab1f89de198d3dbc885d6f89a44883 Author: John Baldwin AuthorDate: 2023-02-15 21:32:52 +0000 Commit: John Baldwin CommitDate: 2023-09-06 21:56:09 +0000 aio: Fix more synchronization issues in aio_biowakeup. - Use atomic_store to set job->error. atomic_set does an or operation, not assignment. - Use refcount_* to manage job->nbio. This ensures proper memory barriers are present so that the last bio won't see a possibly stale value of job->error. - Don't re-read job->error after reading it via atomic_load. Reported by: markj (1) Reviewed by: mjg, markj Differential Revision: https://reviews.freebsd.org/D38611 (cherry picked from commit 98844e99d40a90ae89d84762e07150af3a8f89bd) --- sys/kern/vfs_aio.c | 11 ++++++----- sys/sys/aio.h | 2 +- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/sys/kern/vfs_aio.c b/sys/kern/vfs_aio.c index 0fbe094d8f44..1bdd32ed7969 100644 --- a/sys/kern/vfs_aio.c +++ b/sys/kern/vfs_aio.c @@ -1279,7 +1279,7 @@ aio_qbio(struct proc *p, struct kaiocb *job) } bios = malloc(sizeof(struct bio *) * iovcnt, M_TEMP, M_WAITOK); - atomic_store_int(&job->nbio, iovcnt); + refcount_init(&job->nbio, iovcnt); for (i = 0; i < iovcnt; i++) { struct vm_page** pages; struct bio *bp; @@ -2487,15 +2487,16 @@ aio_biowakeup(struct bio *bp) * error of whichever failed bio completed last. */ if (flags & BIO_ERROR) - atomic_set_int(&job->error, bio_error); + atomic_store_int(&job->error, bio_error); if (opcode & LIO_WRITE) atomic_add_int(&job->outblock, nblks); else atomic_add_int(&job->inblock, nblks); - if (atomic_fetchadd_int(&job->nbio, -1) == 1) { - if (atomic_load_int(&job->error)) - aio_complete(job, -1, job->error); + if (refcount_release(&job->nbio)) { + bio_error = atomic_load_int(&job->error); + if (bio_error != 0) + aio_complete(job, -1, bio_error); else aio_complete(job, atomic_load_long(&job->nbytes), 0); } diff --git a/sys/sys/aio.h b/sys/sys/aio.h index e9c5bdc63c87..a1aa96efed09 100644 --- a/sys/sys/aio.h +++ b/sys/sys/aio.h @@ -149,7 +149,7 @@ struct kaiocb { aio_handle_fn_t *handle_fn; /* (c) backend handle function */ union { /* Backend-specific data fields */ struct { /* BIO backend */ - int nbio; /* Number of remaining bios */ + volatile u_int nbio; /* Number of remaining bios */ int error; /* Worst error of all bios */ long nbytes; /* Bytes completed so far */ };