From nobody Mon Sep 02 08:50:53 2024 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wy2XY6Xztz5Mnxg; Mon, 02 Sep 2024 08:50:53 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wy2XY5QPqz42YJ; Mon, 2 Sep 2024 08:50:53 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1725267053; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=/JLzZTw6sE3FhxkAikcxI9MLfHpGjV4cKMkYvL9LWLs=; b=RDsEzNthuIngbNQPiKLgK+kmnKZKAe3fRSdsPe4LjDSjadzfU+V9gUYdIHTAIZxc4oaTVr Tbwj6rEih+7XR8drqa/elAnX5nsmTLogSslyqwiMazXwULoT0K49OOfvWkIcJD0Pc8SEbc jDdsBSTvS44zjh/l5IqRAV+ETGu6jn6SYQPHSM7n5g9OdqXF93WacE/CGUEg82CZC7DzIJ 1Yh6cMJ3CYj6T0WKfcq7HCjOrvkfuuw/2wM8QT8XBnLX3tFnRPfMpPaAyU1h4w/D+4fyIM 0fV46t0ctOXWKZrQgb4gdQ6ZewcGFvvD6xZXNdn5v6VkuOdWzoKWyxrsvP56YA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1725267053; a=rsa-sha256; cv=none; b=bR8pZZxploI8Oz8Tn3hvln6oS0q5kDqWip54Dc+ja7ksyooUmMauKdF9vNy1ce1x3jDT8L TA8lZJbo1oKup+d3bhnKYbpQIQgWJmImmIoXcHtAC0UFyPkCmw3HQb/jRPL0SmsXYdkUoq x1kbeY+tkq1ePC4yg7YbTCIivTnHkOFc0l7uQtNMfeErq30kUB4GRsoOL2jw6nsQojs27t ekQmDRpTsjTQsQI62IprHid8biDmUIrYk+nJ36/gZJR9T26HUhhAJ4Ug/MDz7GFx6ehrYS rv/nvsnSFx2jDen1Uc12AeURVwYeMO11C/dYpNJXv5VMLD0MAXoLpS5pFORBEA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1725267053; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=/JLzZTw6sE3FhxkAikcxI9MLfHpGjV4cKMkYvL9LWLs=; b=W0aqHQKGoyTGZ5wllbo3UgIaL22ul8l/p95chR1HGmZ+boWOh+lSViROCmOp6XHH6GXWwa a0GqZP+OrGHjf9DmKUZ9ZwonbVE3RKqOv0IzdXCGZ/3hiAaIstUIFMTJWg5SOG3AfSUWHV oHqRSBjL+yp0RgWLRkVgE6vSsvb60AEeriMHTlSJ6MQLFtyylGtqN9Zshr3+0kIbwBxabn dfiNnBXLNCh0+rAXQkApqeP/QKZqE5xo8KNa4t1vQDwEgBlQkqOHw2GeGvrlvtg4yHmDyX wUW8AFictI+sVcI3LQP4ptZrx3jIncdSwbl0BJp8qlHGIAvFhTy3K4YMOPZw5g== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Wy2XY52QLz15LZ; Mon, 2 Sep 2024 08:50:53 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 4828orZx007604; Mon, 2 Sep 2024 08:50:53 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 4828orij007601; Mon, 2 Sep 2024 08:50:53 GMT (envelope-from git) Date: Mon, 2 Sep 2024 08:50:53 GMT Message-Id: <202409020850.4828orij007601@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Andrew Turner Subject: git: 80df53c7fb1c - stable/14 - buf_ring: Use atomic operations with br_cons_tail List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-branches@freebsd.org Sender: owner-dev-commits-src-branches@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 80df53c7fb1cefffff419adbb0767f36a44c27d2 Auto-Submitted: auto-generated The branch stable/14 has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=80df53c7fb1cefffff419adbb0767f36a44c27d2 commit 80df53c7fb1cefffff419adbb0767f36a44c27d2 Author: Andrew Turner AuthorDate: 2024-08-19 09:06:52 +0000 Commit: Andrew Turner CommitDate: 2024-09-02 08:49:38 +0000 buf_ring: Use atomic operations with br_cons_tail Use an atomic operation with a memory barrier loading br_cons_tail from the producer thread and storing to it in the consumer thread. On dequeue we need to read the pointer value from the buf_ring before moving the consumer tail as that indicates the entry is available to be used. The store release atomic operation guarantees this. In the enqueueing thread we then need to use a load acquire atomic operation to ensure writing to this entry can only happen after the tail has been read and checked. Reported by: Ali Saidi Co-developed by: Ali Saidi Reviewed by: markj Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D46152 (cherry picked from commit 44e1cfca417c5ef0db908f3836ec3ba704ef1de2) --- sys/sys/buf_ring.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/sys/sys/buf_ring.h b/sys/sys/buf_ring.h index dec0f971ae44..9aeb5aa19fa8 100644 --- a/sys/sys/buf_ring.h +++ b/sys/sys/buf_ring.h @@ -91,7 +91,7 @@ buf_ring_enqueue(struct buf_ring *br, void *buf) do { prod_head = br->br_prod_head; prod_next = prod_head + 1; - cons_tail = br->br_cons_tail; + cons_tail = atomic_load_acq_32(&br->br_cons_tail); if ((int32_t)(cons_tail + br->br_prod_size - prod_next) < 1) { rmb(); @@ -229,7 +229,7 @@ buf_ring_dequeue_sc(struct buf_ring *br) panic("inconsistent list cons_tail=%d cons_head=%d", br->br_cons_tail, cons_head); #endif - br->br_cons_tail = cons_next; + atomic_store_rel_32(&br->br_cons_tail, cons_next); return (buf); } @@ -257,7 +257,7 @@ buf_ring_advance_sc(struct buf_ring *br) #ifdef DEBUG_BUFRING br->br_ring[cons_head & mask] = NULL; #endif - br->br_cons_tail = cons_next; + atomic_store_rel_32(&br->br_cons_tail, cons_next); } /*