From nobody Sun Sep 29 04:01:57 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XGVrk2xBrz5YGjm; Sun, 29 Sep 2024 04:01:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XGVrj6pmjz4cDj; Sun, 29 Sep 2024 04:01:57 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1727582518; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=bcxMpNpp50wXVmTwf12bGahCaK21tj++NnZevYzZ+wY=; b=cFEaUQ8jDhCYCuDbiEtRkMx+LuWg2b18JRiWhgIjlqbFtR+qmddmz/ixO3v5pSHdrTZv7Z G7W+b+f2tZE+41M0GrJGrINOj3/o8CJ03yGZ3x2ZoIlrtWLg2vH1yLu8m7OiP8+IsGsVS0 qQv4hAjxI1EQeXZwn2AimBlYrb3b4AtlLiX1EIZBgaTxVQDYf08WPGyBDkNBVWBEq4yuLJ zVKoJHYvSIGSvfW60wQ4BW5+k95rNtEbbb1SZAhveHRZkp/jyh90cHxoFvh8sAcYTZSJwt osso/5/2i1BSTb+m5b1ErXfBqg8qaaABZhXpW352JlE2HO6X/lUamnWea6cSoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1727582518; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=bcxMpNpp50wXVmTwf12bGahCaK21tj++NnZevYzZ+wY=; b=kBhpt76s7LPw0iYc1/wqfDiexWH63TYe1uYoVXDOoPUubdPBhNmPDyBUf6PBe6DwD1406A FyrhNLr0JRUFIQIFhMGfZCIcunxZ2d0pzO9oxXiuaP57PnDBMSxkja0YD9qhTPjYmohJc6 7QiUeFqPhYDAu10MsmYRkhIsZfUkMzq/uA6u+940eLfPWrD8gryr8FyH+kziDdS2ftdzpu eiPo+/Y7FhaHQz2fGjjaWC1T7hVLG/RnS9AV3jkRHyx4n0NTBl5KjvQRx5FsSSlTml+j19 gPubxpsxNoltmJacnSMhOePsp2EbauTlmPVwdGNHaM6bBzjJkIs49TEuOwIg/g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1727582518; a=rsa-sha256; cv=none; b=Bpdtg/YDdGP44mwBwdoA/pCVexy7+ZV8xV/kT+aIjzeDAPwi71hRTNs6iQ0jUOJYjZRV4m dayzaNWYdAIkch89BQyI/ncHbXFMSLG2NdMgfEs6jESJq6bfiEk976HwTkEldg3seJtEO5 EzluV7ceIPmBdyn15Ynb5vSB179WXLoREH1zPlhqY+0n4N6UoVhHMDmufE+HIxKgzksSQM 6oSqvaxdtPasqhPkiue5zUELwEJ8a03fgoWeA5mrnaNilX+C0r05q9rfbBsdIpo6ci2sT5 AVt/KV6CTdXQ49Wa9X6gKUWSJnyFj1uRCw15MfnXe1KX9OvUlAnTUQD50DqOxA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4XGVrj6QhtztWF; Sun, 29 Sep 2024 04:01:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 48T41vn6080486; Sun, 29 Sep 2024 04:01:57 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 48T41voB080483; Sun, 29 Sep 2024 04:01:57 GMT (envelope-from git) Date: Sun, 29 Sep 2024 04:01:57 GMT Message-Id: <202409290401.48T41voB080483@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kevin Bowling Subject: git: a648b36e63ca - stable/13 - qlnxe: Fix multiple locking issues List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kbowling X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: a648b36e63ca8c91e32a3383463b7dd50054c58b Auto-Submitted: auto-generated The branch stable/13 has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=a648b36e63ca8c91e32a3383463b7dd50054c58b commit a648b36e63ca8c91e32a3383463b7dd50054c58b Author: Keith Reynolds AuthorDate: 2024-05-28 06:41:05 +0000 Commit: Kevin Bowling CommitDate: 2024-09-29 04:01:08 +0000 qlnxe: Fix multiple locking issues Multiple issues are reported with WITNESS and code inspection of the locking and lock initialization. PR: 278084 (cherry picked from commit 1c45a62a2f667b45ec10a92ad58ff5a34e68b569) --- sys/dev/qlnx/qlnxe/bcm_osal.h | 8 +++---- sys/dev/qlnx/qlnxe/ecore.h | 1 + sys/dev/qlnx/qlnxe/ecore_mcp.c | 48 +++++++++++++++++++++--------------------- sys/dev/qlnx/qlnxe/ecore_mcp.h | 6 +++--- sys/dev/qlnx/qlnxe/qlnx_def.h | 2 +- sys/dev/qlnx/qlnxe/qlnx_os.c | 9 ++++---- sys/dev/qlnx/qlnxe/qlnx_os.h | 4 ++-- 7 files changed, 40 insertions(+), 38 deletions(-) diff --git a/sys/dev/qlnx/qlnxe/bcm_osal.h b/sys/dev/qlnx/qlnxe/bcm_osal.h index 33b78f11ede3..dd5419d40c3a 100644 --- a/sys/dev/qlnx/qlnxe/bcm_osal.h +++ b/sys/dev/qlnx/qlnxe/bcm_osal.h @@ -81,7 +81,7 @@ extern void qlnx_dma_free_coherent(void *ecore_dev, void *v_addr, bus_addr_t phys, uint32_t size); extern void qlnx_link_update(void *p_hwfn); -extern void qlnx_barrier(void *p_hwfn); +extern void qlnx_barrier(void *p_dev); extern void *qlnx_zalloc(uint32_t size); @@ -223,14 +223,14 @@ typedef struct osal_list_t #define OSAL_SPIN_LOCK_ALLOC(p_hwfn, mutex) #define OSAL_SPIN_LOCK_DEALLOC(mutex) mtx_destroy(mutex) #define OSAL_SPIN_LOCK_INIT(lock) {\ - mtx_init(lock, __func__, MTX_NETWORK_LOCK, MTX_SPIN); \ + mtx_init(lock, __func__, "OSAL spin lock", MTX_SPIN); \ } #define OSAL_SPIN_UNLOCK(lock) {\ - mtx_unlock(lock); \ + mtx_unlock_spin(lock); \ } #define OSAL_SPIN_LOCK(lock) {\ - mtx_lock(lock); \ + mtx_lock_spin(lock); \ } #define OSAL_MUTEX_ALLOC(p_hwfn, mutex) diff --git a/sys/dev/qlnx/qlnxe/ecore.h b/sys/dev/qlnx/qlnxe/ecore.h index 5db691b94420..0eaa4c632e05 100644 --- a/sys/dev/qlnx/qlnxe/ecore.h +++ b/sys/dev/qlnx/qlnxe/ecore.h @@ -789,6 +789,7 @@ struct ecore_dev { u8 dp_level; char name[NAME_SIZE]; void *dp_ctx; + void *ha; enum ecore_dev_type type; /* Translate type/revision combo into the proper conditions */ diff --git a/sys/dev/qlnx/qlnxe/ecore_mcp.c b/sys/dev/qlnx/qlnxe/ecore_mcp.c index d94c7382edc5..ab14b1eb5186 100644 --- a/sys/dev/qlnx/qlnxe/ecore_mcp.c +++ b/sys/dev/qlnx/qlnxe/ecore_mcp.c @@ -191,17 +191,17 @@ enum _ecore_status_t ecore_mcp_free(struct ecore_hwfn *p_hwfn) OSAL_FREE(p_hwfn->p_dev, p_hwfn->mcp_info->mfw_mb_cur); OSAL_FREE(p_hwfn->p_dev, p_hwfn->mcp_info->mfw_mb_shadow); - OSAL_SPIN_LOCK(&p_hwfn->mcp_info->cmd_lock); + OSAL_MUTEX_ACQUIRE(&p_hwfn->mcp_info->cmd_lock); OSAL_LIST_FOR_EACH_ENTRY_SAFE(p_cmd_elem, p_tmp, &p_hwfn->mcp_info->cmd_list, list, struct ecore_mcp_cmd_elem) { ecore_mcp_cmd_del_elem(p_hwfn, p_cmd_elem); } - OSAL_SPIN_UNLOCK(&p_hwfn->mcp_info->cmd_lock); + OSAL_MUTEX_RELEASE(&p_hwfn->mcp_info->cmd_lock); #ifdef CONFIG_ECORE_LOCK_ALLOC - OSAL_SPIN_LOCK_DEALLOC(&p_hwfn->mcp_info->cmd_lock); - OSAL_SPIN_LOCK_DEALLOC(&p_hwfn->mcp_info->link_lock); + OSAL_MUTEX_DEALLOC(&p_hwfn->mcp_info->cmd_lock); + OSAL_MUTEX_DEALLOC(&p_hwfn->mcp_info->link_lock); #endif } @@ -308,18 +308,18 @@ enum _ecore_status_t ecore_mcp_cmd_init(struct ecore_hwfn *p_hwfn, /* Initialize the MFW spinlocks */ #ifdef CONFIG_ECORE_LOCK_ALLOC - if (OSAL_SPIN_LOCK_ALLOC(p_hwfn, &p_info->cmd_lock)) { + if (OSAL_MUTEX_LOCK_ALLOC(p_hwfn, &p_info->cmd_lock)) { OSAL_FREE(p_hwfn->p_dev, p_hwfn->mcp_info); return ECORE_NOMEM; } - if (OSAL_SPIN_LOCK_ALLOC(p_hwfn, &p_info->link_lock)) { - OSAL_SPIN_LOCK_DEALLOC(&p_info->cmd_lock); + if (OSAL_MUTEX_ALLOC(p_hwfn, &p_info->link_lock)) { + OSAL_MUTEX_DEALLOC(&p_info->cmd_lock); OSAL_FREE(p_hwfn->p_dev, p_hwfn->mcp_info); return ECORE_NOMEM; } #endif - OSAL_SPIN_LOCK_INIT(&p_info->cmd_lock); - OSAL_SPIN_LOCK_INIT(&p_info->link_lock); + OSAL_MUTEX_INIT(&p_info->cmd_lock); + OSAL_MUTEX_INIT(&p_info->link_lock); OSAL_LIST_INIT(&p_info->cmd_list); @@ -381,7 +381,7 @@ enum _ecore_status_t ecore_mcp_reset(struct ecore_hwfn *p_hwfn, } /* Ensure that only a single thread is accessing the mailbox */ - OSAL_SPIN_LOCK(&p_hwfn->mcp_info->cmd_lock); + OSAL_MUTEX_ACQUIRE(&p_hwfn->mcp_info->cmd_lock); org_mcp_reset_seq = ecore_rd(p_hwfn, p_ptt, MISCS_REG_GENERIC_POR_0); @@ -407,7 +407,7 @@ enum _ecore_status_t ecore_mcp_reset(struct ecore_hwfn *p_hwfn, rc = ECORE_AGAIN; } - OSAL_SPIN_UNLOCK(&p_hwfn->mcp_info->cmd_lock); + OSAL_MUTEX_RELEASE(&p_hwfn->mcp_info->cmd_lock); return rc; } @@ -551,7 +551,7 @@ _ecore_mcp_cmd_and_union(struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt, * The spinlock stays locked until the command is sent. */ - OSAL_SPIN_LOCK(&p_hwfn->mcp_info->cmd_lock); + OSAL_MUTEX_ACQUIRE(&p_hwfn->mcp_info->cmd_lock); if (!ecore_mcp_has_pending_cmd(p_hwfn)) break; @@ -562,7 +562,7 @@ _ecore_mcp_cmd_and_union(struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt, else if (rc != ECORE_AGAIN) goto err; - OSAL_SPIN_UNLOCK(&p_hwfn->mcp_info->cmd_lock); + OSAL_MUTEX_RELEASE(&p_hwfn->mcp_info->cmd_lock); if (ECORE_MB_FLAGS_IS_SET(p_mb_params, CAN_SLEEP)) { OSAL_MSLEEP(msecs); } else { @@ -588,7 +588,7 @@ _ecore_mcp_cmd_and_union(struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt, } __ecore_mcp_cmd_and_union(p_hwfn, p_ptt, p_mb_params, seq_num); - OSAL_SPIN_UNLOCK(&p_hwfn->mcp_info->cmd_lock); + OSAL_MUTEX_RELEASE(&p_hwfn->mcp_info->cmd_lock); /* Wait for the MFW response */ do { @@ -602,7 +602,7 @@ _ecore_mcp_cmd_and_union(struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt, } else { OSAL_UDELAY(usecs); } - OSAL_SPIN_LOCK(&p_hwfn->mcp_info->cmd_lock); + OSAL_MUTEX_ACQUIRE(&p_hwfn->mcp_info->cmd_lock); if (p_cmd_elem->b_is_completed) break; @@ -613,7 +613,7 @@ _ecore_mcp_cmd_and_union(struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt, else if (rc != ECORE_AGAIN) goto err; - OSAL_SPIN_UNLOCK(&p_hwfn->mcp_info->cmd_lock); + OSAL_MUTEX_RELEASE(&p_hwfn->mcp_info->cmd_lock); OSAL_MFW_CMD_PREEMPT(p_hwfn); } while (++cnt < max_retries); @@ -623,9 +623,9 @@ _ecore_mcp_cmd_and_union(struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt, p_mb_params->cmd, p_mb_params->param); ecore_mcp_print_cpu_info(p_hwfn, p_ptt); - OSAL_SPIN_LOCK(&p_hwfn->mcp_info->cmd_lock); + OSAL_MUTEX_ACQUIRE(&p_hwfn->mcp_info->cmd_lock); ecore_mcp_cmd_del_elem(p_hwfn, p_cmd_elem); - OSAL_SPIN_UNLOCK(&p_hwfn->mcp_info->cmd_lock); + OSAL_MUTEX_RELEASE(&p_hwfn->mcp_info->cmd_lock); if (!ECORE_MB_FLAGS_IS_SET(p_mb_params, AVOID_BLOCK)) ecore_mcp_cmd_set_blocking(p_hwfn, true); @@ -634,7 +634,7 @@ _ecore_mcp_cmd_and_union(struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt, } ecore_mcp_cmd_del_elem(p_hwfn, p_cmd_elem); - OSAL_SPIN_UNLOCK(&p_hwfn->mcp_info->cmd_lock); + OSAL_MUTEX_RELEASE(&p_hwfn->mcp_info->cmd_lock); DP_VERBOSE(p_hwfn, ECORE_MSG_SP, "MFW mailbox: response 0x%08x param 0x%08x [after %d.%03d ms]\n", @@ -647,7 +647,7 @@ _ecore_mcp_cmd_and_union(struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt, return ECORE_SUCCESS; err: - OSAL_SPIN_UNLOCK(&p_hwfn->mcp_info->cmd_lock); + OSAL_MUTEX_RELEASE(&p_hwfn->mcp_info->cmd_lock); return rc; } @@ -1439,7 +1439,7 @@ static void ecore_mcp_handle_link_change(struct ecore_hwfn *p_hwfn, u32 status = 0; /* Prevent SW/attentions from doing this at the same time */ - OSAL_SPIN_LOCK(&p_hwfn->mcp_info->link_lock); + OSAL_MUTEX_ACQUIRE(&p_hwfn->mcp_info->link_lock); p_link = &p_hwfn->mcp_info->link_output; OSAL_MEMSET(p_link, 0, sizeof(*p_link)); @@ -1585,7 +1585,7 @@ static void ecore_mcp_handle_link_change(struct ecore_hwfn *p_hwfn, OSAL_LINK_UPDATE(p_hwfn, p_ptt); out: - OSAL_SPIN_UNLOCK(&p_hwfn->mcp_info->link_lock); + OSAL_MUTEX_RELEASE(&p_hwfn->mcp_info->link_lock); } enum _ecore_status_t ecore_mcp_set_link(struct ecore_hwfn *p_hwfn, @@ -1774,7 +1774,7 @@ ecore_mcp_update_bw(struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt) struct public_func shmem_info; u32 resp = 0, param = 0; - OSAL_SPIN_LOCK(&p_hwfn->mcp_info->link_lock); + OSAL_MUTEX_ACQUIRE(&p_hwfn->mcp_info->link_lock); ecore_mcp_get_shmem_func(p_hwfn, p_ptt, &shmem_info, MCP_PF_ID(p_hwfn)); @@ -1787,7 +1787,7 @@ ecore_mcp_update_bw(struct ecore_hwfn *p_hwfn, struct ecore_ptt *p_ptt) ecore_configure_pf_max_bandwidth(p_hwfn->p_dev, p_info->bandwidth_max); - OSAL_SPIN_UNLOCK(&p_hwfn->mcp_info->link_lock); + OSAL_MUTEX_RELEASE(&p_hwfn->mcp_info->link_lock); /* Acknowledge the MFW */ ecore_mcp_cmd(p_hwfn, p_ptt, DRV_MSG_CODE_BW_UPDATE_ACK, 0, &resp, diff --git a/sys/dev/qlnx/qlnxe/ecore_mcp.h b/sys/dev/qlnx/qlnxe/ecore_mcp.h index c94583cdfba3..edb1f9083467 100644 --- a/sys/dev/qlnx/qlnxe/ecore_mcp.h +++ b/sys/dev/qlnx/qlnxe/ecore_mcp.h @@ -51,10 +51,10 @@ struct ecore_mcp_info { /* List for mailbox commands which were sent and wait for a response */ osal_list_t cmd_list; - /* Spinlock used for protecting the access to the mailbox commands list + /* Lock used for protecting the access to the mailbox commands list * and the sending of the commands. */ - osal_spinlock_t cmd_lock; + osal_mutex_t cmd_lock; /* Flag to indicate whether sending a MFW mailbox command is blocked */ bool b_block_cmd; @@ -62,7 +62,7 @@ struct ecore_mcp_info { /* Spinlock used for syncing SW link-changes and link-changes * originating from attention context. */ - osal_spinlock_t link_lock; + osal_mutex_t link_lock; /* Address of the MCP public area */ u32 public_base; diff --git a/sys/dev/qlnx/qlnxe/qlnx_def.h b/sys/dev/qlnx/qlnxe/qlnx_def.h index c63efd19d794..276804c08ab5 100644 --- a/sys/dev/qlnx/qlnxe/qlnx_def.h +++ b/sys/dev/qlnx/qlnxe/qlnx_def.h @@ -391,7 +391,7 @@ struct qlnx_host { int msix_count; - struct mtx hw_lock; + struct sx hw_lock; /* debug */ diff --git a/sys/dev/qlnx/qlnxe/qlnx_os.c b/sys/dev/qlnx/qlnxe/qlnx_os.c index d098694ba19a..8e97321e15dd 100644 --- a/sys/dev/qlnx/qlnxe/qlnx_os.c +++ b/sys/dev/qlnx/qlnxe/qlnx_os.c @@ -785,7 +785,7 @@ qlnx_pci_attach(device_t dev) ha->pci_dev = dev; - mtx_init(&ha->hw_lock, "qlnx_hw_lock", MTX_NETWORK_LOCK, MTX_DEF); + sx_init(&ha->hw_lock, "qlnx_hw_lock"); ha->flags.lock_init = 1; @@ -1232,6 +1232,7 @@ qlnx_init_hw(qlnx_host_t *ha) int rval = 0; struct ecore_hw_prepare_params params; + ha->cdev.ha = ha; ecore_init_struct(&ha->cdev); /* ha->dp_module = ECORE_MSG_PROBE | @@ -1376,7 +1377,7 @@ qlnx_release(qlnx_host_t *ha) pci_release_msi(dev); if (ha->flags.lock_init) { - mtx_destroy(&ha->hw_lock); + sx_destroy(&ha->hw_lock); } if (ha->pci_reg) @@ -5488,11 +5489,11 @@ qlnx_zalloc(uint32_t size) } void -qlnx_barrier(void *p_hwfn) +qlnx_barrier(void *p_dev) { qlnx_host_t *ha; - ha = (qlnx_host_t *)((struct ecore_hwfn *)p_hwfn)->p_dev; + ha = ((struct ecore_dev *) p_dev)->ha; bus_barrier(ha->pci_reg, 0, 0, BUS_SPACE_BARRIER_WRITE); } diff --git a/sys/dev/qlnx/qlnxe/qlnx_os.h b/sys/dev/qlnx/qlnxe/qlnx_os.h index f5711d9e0295..71ccb256ea0c 100644 --- a/sys/dev/qlnx/qlnxe/qlnx_os.h +++ b/sys/dev/qlnx/qlnxe/qlnx_os.h @@ -140,8 +140,8 @@ MALLOC_DECLARE(M_QLNXBUF); /* * Locks */ -#define QLNX_LOCK(ha) mtx_lock(&ha->hw_lock) -#define QLNX_UNLOCK(ha) mtx_unlock(&ha->hw_lock) +#define QLNX_LOCK(ha) sx_xlock(&ha->hw_lock) +#define QLNX_UNLOCK(ha) sx_xunlock(&ha->hw_lock) /* * structure encapsulating a DMA buffer