From nobody Wed Jun 21 10:25:25 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QmKQF5Y8yz4gbN4; Wed, 21 Jun 2023 10:25:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QmKQF53hsz3rQf; Wed, 21 Jun 2023 10:25:25 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1687343125; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=tgBedy3pK+cLjPukqAd9nqRHFsF8qikERhHSUqJEaKM=; b=lBbxnpyoLvPlDifNKo7x+6HFy2fHbHLFPZkJzgpT+iO2tdF0+bBaY1yTLSkyAu+lWTJQFS JWEol0d8HpiJXhxzsrAA4zpQaGtUi3sOAsptF+0XvV7w/+WnC/4ubKw3rw0uqg8vbwVBs5 Dja2O+BirqN2etO6bd4zy8x9DPls+g+z2SUz+oPWv2GfZwwBTQHeX88Ej7YdWR+59hR3Fi HtUolaU8Sws1KcuZyIibxqsfiDzJyyFp83xdO5BQySrrTL6gq1gi+nCWXglzN6gyBEWNRw cyaFjvTpFpipoOMucHtY5SCjfhJ3Evsjh/Li8vpdlWqx4dsRD/Afkj4dACk6Hg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1687343125; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=tgBedy3pK+cLjPukqAd9nqRHFsF8qikERhHSUqJEaKM=; b=l8MkWKMpoL71zdpYw7VGBEUMM9F+KUwHvs0aEuPjzHTZ3J/UlYXXPWGwfSQzLtuPnmnB4S vwo/8LtbbGpvhhPY/YYH9KGe5OVVT4X8SFzyajCTo99EvAaXqX+ERmvIF9oVNvjfPTYar/ QeWO/ZVFywNMLTiUU9Aq5DVHrZUWjrMNV787jq3BVx16qPLJoWKhzDLTYzwcf9lfKSE4Wi VfNAh+ETeUlKup4AAEeCWqyf5x1R3KGYZV/EcDoxChZdomTnF+rgC4U03cTN2N6zOoMUz0 E49ehxvZj3XYZhT2Qn5vSqUjKuHCPzVZdjHpxP5bTj2zmmCr66jV9m/hT5PCBA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1687343125; a=rsa-sha256; cv=none; b=xDnP++0AKhSUZ+uey/yQeFJ5Mf1r8xfZTW3I5tSyr3tfSSqePz1bJnGGhS2XYgkyex6CTs sQnbjpH6oCtXHUnIYEk5JbKBGsQw3od3ZTtcohVSG/SelLFQIf0p4s88nVrp3rYJ6FSVxS 2oMEfwIVSZ5+aBmig6qV8nZ6tbi2+a3LQ6WTw/6GVXk6RPBtLdPyNRLhEvS8Bq3iZvFsG1 Q2he8n/alxnqoHZW8CMyEO49JJXVoFPmCuy5LajX9152JeUUdgrEpnhxgz9A/z/KMHMEOm FqzSmTEmXMqsuRmTkZ7Gua5arnfPX7vpyfuasO+a9vRdq3fvfv+wXtPxXelhaw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4QmKQF41fQzdDr; Wed, 21 Jun 2023 10:25:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 35LAPPdJ026979; Wed, 21 Jun 2023 10:25:25 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 35LAPP5V026978; Wed, 21 Jun 2023 10:25:25 GMT (envelope-from git) Date: Wed, 21 Jun 2023 10:25:25 GMT Message-Id: <202306211025.35LAPP5V026978@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Wei Hu Subject: git: 17050a2b5b07 - main - Hyper-V: vmbus: Prevent load/store reordering when access ring buffer index List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: whu X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 17050a2b5b07160635aaa80b9aa4df4de500b090 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by whu: URL: https://cgit.FreeBSD.org/src/commit/?id=17050a2b5b07160635aaa80b9aa4df4de500b090 commit 17050a2b5b07160635aaa80b9aa4df4de500b090 Author: Wei Hu AuthorDate: 2023-06-21 09:31:46 +0000 Commit: Wei Hu CommitDate: 2023-06-21 10:10:49 +0000 Hyper-V: vmbus: Prevent load/store reordering when access ring buffer index When running VM on ARM64 Hyper-V, we have seen netvsc/hn driver hit assert on reading duplicated network completion packets over vmbus channel or one of the tx channels stalls completely. This seems to caused by processor reordering the instructions when vmbus driver reading or updating its channel ring buffer indexes. Fix this by using load acquire and store release instructions to enforce the order of these memory accesses. PR: 271764 Reported by: Souradeep Chakrabarti Reviewed by: Souradeep Chakrabarti Tested by: whu Sponsored by: Microsoft --- sys/dev/hyperv/vmbus/vmbus_br.c | 31 +++++++++++++++++-------------- sys/dev/hyperv/vmbus/vmbus_brvar.h | 6 ++++-- 2 files changed, 21 insertions(+), 16 deletions(-) diff --git a/sys/dev/hyperv/vmbus/vmbus_br.c b/sys/dev/hyperv/vmbus/vmbus_br.c index 7311f87fd596..c3771f2607b9 100644 --- a/sys/dev/hyperv/vmbus/vmbus_br.c +++ b/sys/dev/hyperv/vmbus/vmbus_br.c @@ -142,8 +142,8 @@ vmbus_rxbr_avail(const struct vmbus_rxbr *rbr) uint32_t rindex, windex; /* Get snapshot */ - rindex = rbr->rxbr_rindex; - windex = rbr->rxbr_windex; + rindex = atomic_load_acq_32(&rbr->rxbr_rindex); + windex = atomic_load_acq_32(&rbr->rxbr_windex); return (rbr->rxbr_dsize - VMBUS_BR_WAVAIL(rindex, windex, rbr->rxbr_dsize)); @@ -289,7 +289,7 @@ vmbus_txbr_need_signal(const struct vmbus_txbr *tbr, uint32_t old_windex) * This is the only case we need to signal when the * ring transitions from being empty to non-empty. */ - if (old_windex == tbr->txbr_rindex) + if (old_windex == atomic_load_acq_32(&tbr->txbr_rindex)) return (TRUE); return (FALSE); @@ -301,8 +301,8 @@ vmbus_txbr_avail(const struct vmbus_txbr *tbr) uint32_t rindex, windex; /* Get snapshot */ - rindex = tbr->txbr_rindex; - windex = tbr->txbr_windex; + rindex = atomic_load_acq_32(&tbr->txbr_rindex); + windex = atomic_load_acq_32(&tbr->txbr_windex); return VMBUS_BR_WAVAIL(rindex, windex, tbr->txbr_dsize); } @@ -427,7 +427,7 @@ vmbus_txbr_write_call(struct vmbus_txbr *tbr, * is copied. */ __compiler_membar(); - tbr->txbr_windex = windex; + atomic_store_rel_32(&tbr->txbr_windex, windex); mtx_unlock_spin(&tbr->txbr_lock); @@ -471,7 +471,7 @@ vmbus_txbr_write(struct vmbus_txbr *tbr, const struct iovec iov[], int iovlen, } /* Save br_windex for later use */ - old_windex = tbr->txbr_windex; + old_windex = atomic_load_acq_32(&tbr->txbr_windex); /* * Copy the scattered channel packet to the TX bufring. @@ -494,7 +494,7 @@ vmbus_txbr_write(struct vmbus_txbr *tbr, const struct iovec iov[], int iovlen, * is copied. */ __compiler_membar(); - tbr->txbr_windex = windex; + atomic_store_rel_32(&tbr->txbr_windex, windex); mtx_unlock_spin(&tbr->txbr_lock); @@ -557,7 +557,8 @@ vmbus_rxbr_peek(struct vmbus_rxbr *rbr, void *data, int dlen) mtx_unlock_spin(&rbr->rxbr_lock); return (EAGAIN); } - vmbus_rxbr_copyfrom(rbr, rbr->rxbr_rindex, data, dlen); + vmbus_rxbr_copyfrom(rbr, + atomic_load_acq_32(&rbr->rxbr_rindex), data, dlen); mtx_unlock_spin(&rbr->rxbr_lock); @@ -622,10 +623,11 @@ vmbus_rxbr_idxadv_peek(struct vmbus_rxbr *rbr, void *data, int dlen, rindex = VMBUS_BR_IDXINC(rbr->rxbr_rindex, idx_adv + sizeof(uint64_t), br_dsize); __compiler_membar(); - rbr->rxbr_rindex = rindex; + atomic_store_rel_32(&rbr->rxbr_rindex, rindex); } - vmbus_rxbr_copyfrom(rbr, rbr->rxbr_rindex, data, dlen); + vmbus_rxbr_copyfrom(rbr, + atomic_load_acq_32(&rbr->rxbr_rindex), data, dlen); mtx_unlock_spin(&rbr->rxbr_lock); @@ -667,7 +669,7 @@ vmbus_rxbr_idxadv(struct vmbus_rxbr *rbr, uint32_t idx_adv, rindex = VMBUS_BR_IDXINC(rbr->rxbr_rindex, idx_adv + sizeof(uint64_t), br_dsize); __compiler_membar(); - rbr->rxbr_rindex = rindex; + atomic_store_rel_32(&rbr->rxbr_rindex, rindex); mtx_unlock_spin(&rbr->rxbr_lock); @@ -700,7 +702,8 @@ vmbus_rxbr_read(struct vmbus_rxbr *rbr, void *data, int dlen, uint32_t skip) /* * Copy channel packet from RX bufring. */ - rindex = VMBUS_BR_IDXINC(rbr->rxbr_rindex, skip, br_dsize); + rindex = VMBUS_BR_IDXINC(atomic_load_acq_32(&rbr->rxbr_rindex), + skip, br_dsize); rindex = vmbus_rxbr_copyfrom(rbr, rindex, data, dlen); /* @@ -712,7 +715,7 @@ vmbus_rxbr_read(struct vmbus_rxbr *rbr, void *data, int dlen, uint32_t skip) * Update the read index _after_ the channel packet is fetched. */ __compiler_membar(); - rbr->rxbr_rindex = rindex; + atomic_store_rel_32(&rbr->rxbr_rindex, rindex); mtx_unlock_spin(&rbr->rxbr_lock); diff --git a/sys/dev/hyperv/vmbus/vmbus_brvar.h b/sys/dev/hyperv/vmbus/vmbus_brvar.h index 95bf4338ff1c..0edcc61ae5e1 100644 --- a/sys/dev/hyperv/vmbus/vmbus_brvar.h +++ b/sys/dev/hyperv/vmbus/vmbus_brvar.h @@ -99,14 +99,16 @@ static __inline bool vmbus_txbr_empty(const struct vmbus_txbr *tbr) { - return (tbr->txbr_windex == tbr->txbr_rindex ? true : false); + return (atomic_load_acq_32(&tbr->txbr_windex) == + atomic_load_acq_32(&tbr->txbr_rindex) ? true : false); } static __inline bool vmbus_rxbr_empty(const struct vmbus_rxbr *rbr) { - return (rbr->rxbr_windex == rbr->rxbr_rindex ? true : false); + return (atomic_load_acq_32(&rbr->rxbr_windex) == + atomic_load_acq_32(&rbr->rxbr_rindex) ? true : false); } static __inline int