From nobody Mon Sep 02 09:12:50 2024 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wy31t2Vtcz5Mr5v; Mon, 02 Sep 2024 09:12:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wy31t1SLnz4Csh; Mon, 2 Sep 2024 09:12:50 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1725268370; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=GzauxNYGekcGYl1qzNlmAdBHCTy8I3GO7dgJjGnpcaI=; b=fUn+OXiNK3b6LmZsquEwHoq+I8cQcopjNX4LrVx+DAcCXpYUZ71Pao2B06KRVkzU0EyKV8 1WSymT/ai2kWOOYG0Yv2QmeYleVjmpgaruqK/4sY32agtqFJ3xohfrnOTsY0F/HM5UjkuT NPZoUcoE/hFqJaqSCAb0/xWXUnQn3wo8UrCKhziiK5KwFTkvxMnmhZMB/rsvJrc3qh7ghF 8LRtFQJBCDnSY1I1CoFmAyUNE2vWrBMTZc/OsttJymLW0OM7fiCNjcBW80GlXg/ADXZz3A 0nQGZ35aG7FpTgAQyeQUl2Dme9i9kPoQUR+FWR6XU0bM7hWNoO/+2KFSZorS7Q== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1725268370; a=rsa-sha256; cv=none; b=qCJfWNH7xBilZoZLN+iAyknpIHwz8E5kTtjtdizKY+z06bY0eOXFy8ehubsY3BwOq9fxhz myWufQCtlHmnVEeVg6CoZK+uQ3p3Qta+aK1T9dWmI1aeCWIU65h9KADLosNlYwwpWrVgid ZpfYEigL5Uqp1qsMif2bUelimgihPRolSNSkOCBrTnF/72XA0F9oCP4CkZrBhUvaMJprxO r3jbN7zOBtHf4du3NlmEOAfzHn0LLh9Tzn/sx3pjxQkOZNeHt6cIGUslA35Un8eFg+f57/ /8IHXYggrbdJBaG3dwdGajY9w1wAOmIjutTaa8P0FTv+4eDkv0lHHJl9ckGB2g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1725268370; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=GzauxNYGekcGYl1qzNlmAdBHCTy8I3GO7dgJjGnpcaI=; b=krBBP1b5KtulWXEjnRPGH7l17nlbsyQoJWruuvb4zyvl6gyvt0QzENBNpTnLBYUxbHnqo0 twfijZ8cs4DxgOxIBe9LkGlqt5zcHnpA7OHj5U1/2LwIvHwSH6kqd2iWAEQjzbmX0ETurl 3NKaASmzwuRfyPtGStA28Bufun7wq76mhhhv9neDAnpOi3I/66bNApxCRI4cyyJqi05DdO 0aWBfr8H7tJqUWIL2RehteTeyo313037zxOLCrX4hjeOBkY/Nq8VyNFqnvhus2eFROUUa6 ykVo58cJcICcY3nNQoUMSR2g3KfZGRVLiNqJnVjOGurnthJ7ITIL51Ct9+2hpw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Wy31t10wPz16Dp; Mon, 2 Sep 2024 09:12:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 4829Coru050086; Mon, 2 Sep 2024 09:12:50 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 4829Cort050083; Mon, 2 Sep 2024 09:12:50 GMT (envelope-from git) Date: Mon, 2 Sep 2024 09:12:50 GMT Message-Id: <202409020912.4829Cort050083@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Andrew Turner Subject: git: 39757e10689d - stable/13 - buf_ring: Ensure correct ordering of loads List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-branches@freebsd.org Sender: owner-dev-commits-src-branches@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 39757e10689d44036b5d10de41d766641e59eec0 Auto-Submitted: auto-generated The branch stable/13 has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=39757e10689d44036b5d10de41d766641e59eec0 commit 39757e10689d44036b5d10de41d766641e59eec0 Author: Andrew Turner AuthorDate: 2024-08-19 09:07:26 +0000 Commit: Andrew Turner CommitDate: 2024-09-02 09:11:57 +0000 buf_ring: Ensure correct ordering of loads When enqueueing on an architecture with a weak memory model ensure loading br->br_prod_head and br->br_cons_tail are ordered correctly. If br_cons_tail is loaded first then other threads may perform a dequeue and enqueue before br_prod_head is loaded. This will mean the tail is one less than it should be and the code under the prod_next == cons_tail check could incorrectly be skipped. buf_ring_dequeue_mc has the same issue with br->br_prod_tail and br->br_cons_head so needs the same fix. Reported by: Ali Saidi Co-developed by: Ali Saidi Reviewed by: imp, kib, markj Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D46155 (cherry picked from commit fe2445f47d027c73aa7266669e7d94b70d3949a4) --- sys/sys/buf_ring.h | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/sys/sys/buf_ring.h b/sys/sys/buf_ring.h index 66e1e55bc5e9..512f20dc13e2 100644 --- a/sys/sys/buf_ring.h +++ b/sys/sys/buf_ring.h @@ -89,7 +89,17 @@ buf_ring_enqueue(struct buf_ring *br, void *buf) #endif critical_enter(); do { - prod_head = br->br_prod_head; + /* + * br->br_prod_head needs to be read before br->br_cons_tail. + * If not then we could perform the dequeue and enqueue + * between reading br_cons_tail and reading br_prod_head. This + * could give us values where br_cons_head == br_prod_tail + * (after masking). + * + * To work around this us a load acquire. This is just to + * ensure ordering within this thread. + */ + prod_head = atomic_load_acq_32(&br->br_prod_head); prod_next = prod_head + 1; cons_tail = atomic_load_acq_32(&br->br_cons_tail); @@ -137,7 +147,12 @@ buf_ring_dequeue_mc(struct buf_ring *br) critical_enter(); mask = br->br_cons_mask; do { - cons_head = br->br_cons_head; + /* + * As with buf_ring_enqueue ensure we read the head before + * the tail. If we read them in the wrong order we may + * think the bug_ring is full when it is empty. + */ + cons_head = atomic_load_acq_32(&br->br_cons_head); cons_next = cons_head + 1; prod_tail = atomic_load_acq_32(&br->br_prod_tail);