From nobody Thu Jun 06 10:39:23 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Vw16M68BHz5MfwS; Thu, 06 Jun 2024 10:39:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Vw16M2RqTz4j4T; Thu, 6 Jun 2024 10:39:23 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1717670363; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=WBNuN/hSQWjzTxYZt2+JsSzGFs/rX5TuHPrf03+MomY=; b=e24DYxwplEKE34feeWxi0KvI4HcuFH+HXTG3ExHQ9dngsvtki0a7iHoaWDCW+6LhY/T7mq PPRS2pCE/rkTeNavnsNfUOqxnvwrzcLzdErYe/qUg6aL6f6qp9fd/1DRVr46dfDv2miYsu 9I6qn1D0k029JD7BylcwSH/+jPdoaxozZN0pL7hJkpk4TvfBL3vyFB4goM5CBK9pZoPakW aqH0U8eZ9NiK8T0ArMWXL7r9ixk4uTrfcFuJhZlCNAopc7sKh6WRfsTxSB95THNuUQ/F3P IH/r9Pu2uEArqLi8N5pT+Uenc3eGGxIjEmqCK6U8lLR9F+9BTZG8l6unI+eOpg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1717670363; a=rsa-sha256; cv=none; b=WOv2/kKTQuygWeguh7ZZDgsr516ax2syWFmFuZO5ZN+a3ne0468d0rZhwIsuqIf+HEhuMz Rcbk6iKiMFVmfDmXuSesORU7ZUw/hFeoPEpCHrIW5isbzVERLSAkPkjp6eUBjET//DLKNB vKQ404s403n6eqcREvhvxfnnazdkVwmOCWgFeJt7jzbf6VnlmA9kLeuviv2B8oiDVINH6l DP0OxQGb3SLrBzP7PgVOuCafGYR7uh5visz8ng2Yh4wfGQy+j3T9X2JL0qKy+jG8zOai2H MbeifXfQQ4oW32Zjvw8FfMHWbt7WHtBLQe0JUiS/KVSJCqWumFE2U13fDASYKw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1717670363; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=WBNuN/hSQWjzTxYZt2+JsSzGFs/rX5TuHPrf03+MomY=; b=Oz6H5+bH2J9tM8gnq+XQ0GDbhOFVrXuERzxXtUhl57+8absCdTrDzoC3pi0URe7t/Ro1e3 afuM75qzOlwbnK9RcWHiYJa9x+PzmWy6OlUtfOEzV7gXThMw8VX8YefomcHfdPp9K99EoM OH5g/NoX3j0zUksXBW0Mt8nNzueuxr8ZkYItJb7/Jc1cYtbi9CYeDlvJdaNbT5KLWoLwoZ nCoOreekk31zzT6zagCVfLWZPJsYiVxUyHqPDBvW1CkSh+i1MKEf8Ki2ISmZKQCOnXyanr 6kcmnmZDLGUQZTZItjqUu6FxDMNOVT6V3mIzIO4IbAZkco7fseW2QEGg+13SHA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Vw16M189FzjLj; Thu, 6 Jun 2024 10:39:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 456AdNjm089627; Thu, 6 Jun 2024 10:39:23 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 456AdNuD089624; Thu, 6 Jun 2024 10:39:23 GMT (envelope-from git) Date: Thu, 6 Jun 2024 10:39:23 GMT Message-Id: <202406061039.456AdNuD089624@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Sumit Saxena Subject: git: 701d776c98e0 - main - mpi3mr: poll reply queue and add MPI3MR_DEV_REMOVE_HS_COMPLETED flag List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: ssaxena X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 701d776c98e015160c2d0e3f4fdaf980d1d5dc83 Auto-Submitted: auto-generated The branch main has been updated by ssaxena: URL: https://cgit.FreeBSD.org/src/commit/?id=701d776c98e015160c2d0e3f4fdaf980d1d5dc83 commit 701d776c98e015160c2d0e3f4fdaf980d1d5dc83 Author: Chandrakanth patil AuthorDate: 2024-03-14 17:47:06 +0000 Commit: Sumit Saxena CommitDate: 2024-06-06 10:39:14 +0000 mpi3mr: poll reply queue and add MPI3MR_DEV_REMOVE_HS_COMPLETED flag An outstanding IO counter per target check has been added before deleting the target from the OS which will poll the reply queue if there are any outstanding IOs are found. A new flag, named "MPI3MR_DEV_REMOVE_HS_COMPLETED," is added. If a remove event for a target occurs and before the deletion of the target resource if the add event for another target arrives reusing the same target ID then this flag will prevent the removal of the target reference. This flag ensures synchronization between the interrupt top and bottom half during target removal and addition events. Reviewed by: imp Approved by: imp Differential revision: https://reviews.freebsd.org/D44423 --- sys/dev/mpi3mr/mpi3mr.c | 19 ++++++++----------- sys/dev/mpi3mr/mpi3mr.h | 3 +-- sys/dev/mpi3mr/mpi3mr_cam.c | 27 ++++++++++++++++++--------- 3 files changed, 27 insertions(+), 22 deletions(-) diff --git a/sys/dev/mpi3mr/mpi3mr.c b/sys/dev/mpi3mr/mpi3mr.c index faa8b52bbb50..72abe76ad085 100644 --- a/sys/dev/mpi3mr/mpi3mr.c +++ b/sys/dev/mpi3mr/mpi3mr.c @@ -3477,6 +3477,7 @@ static void mpi3mr_dev_rmhs_complete_iou(struct mpi3mr_softc *sc, { U16 cmd_idx = drv_cmd->host_tag - MPI3MR_HOSTTAG_DEVRMCMD_MIN; struct delayed_dev_rmhs_node *delayed_dev_rmhs = NULL; + struct mpi3mr_target *tgtdev = NULL; mpi3mr_dprint(sc, MPI3MR_EVENT, "%s :dev_rmhs_iouctrl_complete:handle(0x%04x), ioc_status(0x%04x), loginfo(0x%08x)\n", @@ -3497,6 +3498,13 @@ static void mpi3mr_dev_rmhs_complete_iou(struct mpi3mr_softc *sc, "%s :dev removal handshake failed after all retries: handle(0x%04x)\n", __func__, drv_cmd->dev_handle); } else { + mtx_lock_spin(&sc->target_lock); + TAILQ_FOREACH(tgtdev, &sc->cam_sc->tgt_list, tgt_next) { + if (tgtdev->dev_handle == drv_cmd->dev_handle) + tgtdev->state = MPI3MR_DEV_REMOVE_HS_COMPLETED; + } + mtx_unlock_spin(&sc->target_lock); + mpi3mr_dprint(sc, MPI3MR_INFO, "%s :dev removal handshake completed successfully: handle(0x%04x)\n", __func__, drv_cmd->dev_handle); @@ -3604,18 +3612,7 @@ static void mpi3mr_dev_rmhs_send_tm(struct mpi3mr_softc *sc, U16 handle, U8 retrycount = 5; struct mpi3mr_drvr_cmd *drv_cmd = cmdparam; struct delayed_dev_rmhs_node *delayed_dev_rmhs = NULL; - struct mpi3mr_target *tgtdev = NULL; - mtx_lock_spin(&sc->target_lock); - TAILQ_FOREACH(tgtdev, &sc->cam_sc->tgt_list, tgt_next) { - if ((tgtdev->dev_handle == handle) && - (iou_rc == MPI3_CTRL_OP_REMOVE_DEVICE)) { - tgtdev->state = MPI3MR_DEV_REMOVE_HS_STARTED; - break; - } - } - mtx_unlock_spin(&sc->target_lock); - if (drv_cmd) goto issue_cmd; do { diff --git a/sys/dev/mpi3mr/mpi3mr.h b/sys/dev/mpi3mr/mpi3mr.h index d93c53b286cb..1cf6718d9047 100644 --- a/sys/dev/mpi3mr/mpi3mr.h +++ b/sys/dev/mpi3mr/mpi3mr.h @@ -446,8 +446,7 @@ enum mpi3mr_cmd_state { enum mpi3mr_target_state { MPI3MR_DEV_CREATED = 1, - MPI3MR_DEV_REMOVE_HS_STARTED = 2, - MPI3MR_DEV_DELETED = 3, + MPI3MR_DEV_REMOVE_HS_COMPLETED = 2, }; struct mpi3mr_cmd { diff --git a/sys/dev/mpi3mr/mpi3mr_cam.c b/sys/dev/mpi3mr/mpi3mr_cam.c index b842e2a05bda..b8e4e350dfa1 100644 --- a/sys/dev/mpi3mr/mpi3mr_cam.c +++ b/sys/dev/mpi3mr/mpi3mr_cam.c @@ -1822,6 +1822,17 @@ int mpi3mr_remove_device_from_os(struct mpi3mr_softc *sc, U16 handle) target->flags |= MPI3MRSAS_TARGET_INREMOVAL; + if (mpi3mr_atomic_read(&target->outstanding)) { + mpi3mr_dprint(sc, MPI3MR_ERROR, "there are [%2d] outstanding IOs on target: %d" + "Poll reply queue once\n", mpi3mr_atomic_read(&target->outstanding), + target->per_id); + mpi3mr_poll_pend_io_completions(sc); + if (mpi3mr_atomic_read(&target->outstanding)) + mpi3mr_dprint(sc, MPI3MR_ERROR, "[%2d] outstanding IOs present on target: %d" + "despite poll\n", mpi3mr_atomic_read(&target->outstanding), + target->per_id); + } + while (mpi3mr_atomic_read(&target->outstanding) && (i < 30)) { i++; if (!(i % 2)) { @@ -1848,18 +1859,16 @@ out: void mpi3mr_remove_device_from_list(struct mpi3mr_softc *sc, struct mpi3mr_target *target, bool must_delete) { + if ((must_delete == false) && + (target->state != MPI3MR_DEV_REMOVE_HS_COMPLETED)) + return; + mtx_lock_spin(&sc->target_lock); - if ((target->state == MPI3MR_DEV_REMOVE_HS_STARTED) || - (must_delete == true)) { - TAILQ_REMOVE(&sc->cam_sc->tgt_list, target, tgt_next); - target->state = MPI3MR_DEV_DELETED; - } + TAILQ_REMOVE(&sc->cam_sc->tgt_list, target, tgt_next); mtx_unlock_spin(&sc->target_lock); - if (target->state == MPI3MR_DEV_DELETED) { - free(target, M_MPI3MR); - target = NULL; - } + free(target, M_MPI3MR); + target = NULL; return; }