From nobody Mon Aug 19 11:03:32 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WnV846hNMz5SpCd; Mon, 19 Aug 2024 11:03:32 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WnV844f3lz4SGp; Mon, 19 Aug 2024 11:03:32 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1724065412; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=RmKZqYGPIyhgUWKfWNO/QMMpE4YKFyGaKKrB7LzIMfk=; b=l5e6/veOYkNOGdzyTKujo80sHrHuehvJDzmCCCjp7GWDyMcEBTKogyV5CqHzR3IYIgyc4K 31MlAnjU5Vpio4E4cYVsWAV0b1ro836VB5T+LdvJxxIKwXz7IGpXLUlL9k3K4J56kNi1Re qOBAyTJ4goP4WNWNx84UwY35A4ITwBfaNEQpZ8XmA/Ys1HXQhVSK+Iy7ZphrFqaErbc+GD STFcqMZfKglQLCwa3sY+Y27UXig/k9ySIEArWJ8acDDIUqoLW+re4KldytYhifesRM1b3t UC90OHIjTNNDHt+CLUrqn+4WOzzv2SQye/eGYZwDS5WWztqjje90Qdu36YRe4Q== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1724065412; a=rsa-sha256; cv=none; b=ovh+2E7FebX+8endtAAo88dBaMLHjXQ+bDdNoWtWFq0LYa005rXUHFArij42dz37ho3jpO WVAZR5sETT4DySSI4AFo8Ob2rxIJ61WIk7AftR/CtsxWsyeBfwmX2fT5YVBEixAiX51uLO tVXQziYOWNMCcsnzC6BpJ8i075cYsG2n0Fw5MHCQ4shWKPuyqPOr6f/4e2VwSvHs1HKvZX MlxUujhr/bY59X0Wz/OO07dXcavVbsugbEn+m47nbKyg9sybRtQa6En0anXFeu81bw1bWT HT3rjJfYpwM8ALLGWnd4WCBrjuOqvDpR9n1JmRHr0QiVbp+pHxNsGCv4EHGuBw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1724065412; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=RmKZqYGPIyhgUWKfWNO/QMMpE4YKFyGaKKrB7LzIMfk=; b=FbcT8xTjyCDQWrvhYjSNnYJqD0pOuyjanPY71OFY9SIZqM+diTNhjsRvhbaSkMuaXunkdv MsDukYSprk0uS2Q3CSLbOX7jIRLTPjk+ks1dXRqCS20dJAtuW9WMTTwgAolqI20afCqQBb OwfOdIqanCzlAV/aozVGLCEGtJu1i8AvR5I6JhmWF/Gi/CInrKzFm/8T95vb9ZmhF0o8vl hEdhQ0djzMz3YDS2ap34hxvEDFCNMBL95ga2fM5z/i+7px103tNob2yYqw/KIc0IBdkuLD Npqh5vAj3bnNLfCO8wauiiH+nHCLWS70CG58TolQXutAzAf7+qALfMN6bj7z4Q== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WnV843rBqzF05; Mon, 19 Aug 2024 11:03:32 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 47JB3WJ1037786; Mon, 19 Aug 2024 11:03:32 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 47JB3W9A037783; Mon, 19 Aug 2024 11:03:32 GMT (envelope-from git) Date: Mon, 19 Aug 2024 11:03:32 GMT Message-Id: <202408191103.47JB3W9A037783@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Andrew Turner Subject: git: fe2445f47d02 - main - buf_ring: Ensure correct ordering of loads List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: fe2445f47d027c73aa7266669e7d94b70d3949a4 Auto-Submitted: auto-generated The branch main has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=fe2445f47d027c73aa7266669e7d94b70d3949a4 commit fe2445f47d027c73aa7266669e7d94b70d3949a4 Author: Andrew Turner AuthorDate: 2024-08-19 09:07:26 +0000 Commit: Andrew Turner CommitDate: 2024-08-19 10:53:11 +0000 buf_ring: Ensure correct ordering of loads When enqueueing on an architecture with a weak memory model ensure loading br->br_prod_head and br->br_cons_tail are ordered correctly. If br_cons_tail is loaded first then other threads may perform a dequeue and enqueue before br_prod_head is loaded. This will mean the tail is one less than it should be and the code under the prod_next == cons_tail check could incorrectly be skipped. buf_ring_dequeue_mc has the same issue with br->br_prod_tail and br->br_cons_head so needs the same fix. Reported by: Ali Saidi Co-developed by: Ali Saidi Reviewed by: imp, kib, markj Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D46155 --- sys/sys/buf_ring.h | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/sys/sys/buf_ring.h b/sys/sys/buf_ring.h index 66e1e55bc5e9..512f20dc13e2 100644 --- a/sys/sys/buf_ring.h +++ b/sys/sys/buf_ring.h @@ -89,7 +89,17 @@ buf_ring_enqueue(struct buf_ring *br, void *buf) #endif critical_enter(); do { - prod_head = br->br_prod_head; + /* + * br->br_prod_head needs to be read before br->br_cons_tail. + * If not then we could perform the dequeue and enqueue + * between reading br_cons_tail and reading br_prod_head. This + * could give us values where br_cons_head == br_prod_tail + * (after masking). + * + * To work around this us a load acquire. This is just to + * ensure ordering within this thread. + */ + prod_head = atomic_load_acq_32(&br->br_prod_head); prod_next = prod_head + 1; cons_tail = atomic_load_acq_32(&br->br_cons_tail); @@ -137,7 +147,12 @@ buf_ring_dequeue_mc(struct buf_ring *br) critical_enter(); mask = br->br_cons_mask; do { - cons_head = br->br_cons_head; + /* + * As with buf_ring_enqueue ensure we read the head before + * the tail. If we read them in the wrong order we may + * think the bug_ring is full when it is empty. + */ + cons_head = atomic_load_acq_32(&br->br_cons_head); cons_next = cons_head + 1; prod_tail = atomic_load_acq_32(&br->br_prod_tail);