From nobody Thu Oct 31 16:01:29 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XfTJ970Xnz5bwgs; Thu, 31 Oct 2024 16:01:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XfTJ921Xpz4T4P; Thu, 31 Oct 2024 16:01:29 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1730390489; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ddwXBCE3u+YKoYXx6pZlcz/X7+xVUiyo87vrgDHPxz0=; b=RW9WWnjGdxuSFxvJIbfld0YXVY9PDCz4iDr7BZ2mlEkSncAQgjsZB56gF1d2a7Gw/bKlw4 k/AlorsGKFbQRpKK0HHBp2pp0l45XOjqH3lXmKQFsTUX42TmvIqqnHGbRArhXXEacE6ADN tEDjFz6hu5ruDAYhnVSaCH8XC56/QsmM3utj7gMid/Tkazw+UdbXy7l2cv1hv15mjv6QaW WKv1WVDDMNsZ+FJZQTpSc4tnLroZVhvVZr+QK3E3nPI6KXwuzkKIVkIpF2eibeo7IWBC5w Bx6yl+0OtqOszv+jRVYKQ2kWbSgaBFpfyESt+SfYAubrk3ebvpjdVr/weBdz4w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1730390489; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ddwXBCE3u+YKoYXx6pZlcz/X7+xVUiyo87vrgDHPxz0=; b=LuU/PyAUWapwRKzHkQXGH0TCC5Nu3ieKmwZFJgY9Zv+YKE+OIh0kNVBClg2ggXgIS4AloM KMMaSFcvDA9/3cCMzpTVQzWHZPl4IK7XxsPz/Yo/td5SDbxh8IDxekQ6P36CQDvNZqCvVZ 6UDdBhYyHZoEY/2hciBvM33/c2p3jDYCRM6a5VvLF2vxBu5QXtKBY9J/d6wA4HKqECOIyx x9dDeIdSOmcEVfpgSV0kaFuVNY9Eolf8XqIOzz9GuXQqCu0LKeKM80HdOZW7vfXiZ/2AQx 6hzd29wmNhUwFXfPatXN5lUA2Yt89NQW2IW69qPBl8qPEBXcS96P+Q1BfgOesg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1730390489; a=rsa-sha256; cv=none; b=YCFbtBPHDExA9upgRYuRJXOIt+AA6WVzR0iVnEz7SjHtSQ+YK2034dl1yVTN8be6Ox2j5L tewzhB3iHEVV6cyBaZMIT4p4hRGpnFLPlP0PBNqwrzBwfqAoO8Y+h3aomrIKwCElAmL3iU +moXxmMbP018ACHh+y+Yorik1MvZWX3N2UTFsyK9WeGcgKpERd3F5e51Vn+o4wc6Jo5vma vXiGvQ2gbobbzEcpuU+9W9qC9OMz+f2p/NPfg5XhhFnn6Rt0Es6laG9Lee6NEj7ohGlNxt zIjjPE/NSnCguW9oSGExdLMBM/7m8FBZSl9zhWlPIIW37tD8P8f0d3HMm2optA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4XfTJ91d93zjVp; Thu, 31 Oct 2024 16:01:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 49VG1TCv070208; Thu, 31 Oct 2024 16:01:29 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 49VG1Tbp070205; Thu, 31 Oct 2024 16:01:29 GMT (envelope-from git) Date: Thu, 31 Oct 2024 16:01:29 GMT Message-Id: <202410311601.49VG1Tbp070205@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Osama Abboud Subject: git: a0594d1f65d6 - stable/14 - ena: Add reset reason for missing admin interrupt List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: osamaabb X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: a0594d1f65d67b2ff91cf9c0caa94cdebae7e04e Auto-Submitted: auto-generated The branch stable/14 has been updated by osamaabb: URL: https://cgit.FreeBSD.org/src/commit/?id=a0594d1f65d67b2ff91cf9c0caa94cdebae7e04e commit a0594d1f65d67b2ff91cf9c0caa94cdebae7e04e Author: Osama Abboud AuthorDate: 2024-08-07 06:24:19 +0000 Commit: Osama Abboud CommitDate: 2024-10-31 14:54:10 +0000 ena: Add reset reason for missing admin interrupt There can be cases when we trigger reset if an admin interrupt is missing. In order to identify this use-case specifically, this commit adds a new reset reason. Approved by: cperciva (mentor) Sponsored by: Amazon, Inc. (cherry picked from commit 274319acb48424958242d55e1b0c7d4528da7f70) --- sys/dev/ena/ena.c | 13 +++++++++++-- sys/dev/ena/ena.h | 5 ++++- sys/dev/ena/ena_sysctl.c | 4 ++++ 3 files changed, 19 insertions(+), 3 deletions(-) diff --git a/sys/dev/ena/ena.c b/sys/dev/ena/ena.c index 3f3a4946ccca..36e9ac15e8ff 100644 --- a/sys/dev/ena/ena.c +++ b/sys/dev/ena/ena.c @@ -3029,6 +3029,7 @@ static void check_for_missing_keep_alive(struct ena_adapter *adapter) { sbintime_t timestamp, time; + enum ena_regs_reset_reason_types reset_reason = ENA_REGS_RESET_KEEP_ALIVE_TO; if (adapter->wd_active == 0) return; @@ -3040,7 +3041,10 @@ check_for_missing_keep_alive(struct ena_adapter *adapter) time = getsbinuptime() - timestamp; if (unlikely(time > adapter->keep_alive_timeout)) { ena_log(adapter->pdev, ERR, "Keep alive watchdog timeout.\n"); - ena_trigger_reset(adapter, ENA_REGS_RESET_KEEP_ALIVE_TO); + if (ena_com_aenq_has_keep_alive(adapter->ena_dev)) + reset_reason = ENA_REGS_RESET_MISSING_ADMIN_INTERRUPT; + + ena_trigger_reset(adapter, reset_reason); } } @@ -3048,10 +3052,15 @@ check_for_missing_keep_alive(struct ena_adapter *adapter) static void check_for_admin_com_state(struct ena_adapter *adapter) { + enum ena_regs_reset_reason_types reset_reason = ENA_REGS_RESET_ADMIN_TO; if (unlikely(ena_com_get_admin_running_state(adapter->ena_dev) == false)) { ena_log(adapter->pdev, ERR, "ENA admin queue is not in running state!\n"); - ena_trigger_reset(adapter, ENA_REGS_RESET_ADMIN_TO); + counter_u64_add(adapter->dev_stats.admin_q_pause, 1); + if (ena_com_get_missing_admin_interrupt(adapter->ena_dev)) + reset_reason = ENA_REGS_RESET_MISSING_ADMIN_INTERRUPT; + + ena_trigger_reset(adapter, reset_reason); } } diff --git a/sys/dev/ena/ena.h b/sys/dev/ena/ena.h index b747736224d8..1a436a702ba1 100644 --- a/sys/dev/ena/ena.h +++ b/sys/dev/ena/ena.h @@ -391,6 +391,8 @@ struct ena_stats_dev { counter_u64_t missing_intr; counter_u64_t tx_desc_malformed; counter_u64_t rx_desc_malformed; + counter_u64_t missing_admin_interrupt; + counter_u64_t admin_to; }; struct ena_hw_stats { @@ -542,7 +544,7 @@ struct ena_reset_stats_offset { static const struct ena_reset_stats_offset resets_to_stats_offset_map[ENA_REGS_RESET_LAST] = { ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_KEEP_ALIVE_TO, wd_expired), - ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_ADMIN_TO, admin_q_pause), + ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_ADMIN_TO, admin_to), ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_OS_TRIGGER, os_trigger), ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_MISS_TX_CMPL, missing_tx_cmpl), ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_INV_RX_REQ_ID, bad_rx_req_id), @@ -552,6 +554,7 @@ static const struct ena_reset_stats_offset resets_to_stats_offset_map[ENA_REGS_R ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_MISS_INTERRUPT, missing_intr), ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_TX_DESCRIPTOR_MALFORMED, tx_desc_malformed), ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_RX_DESCRIPTOR_MALFORMED, rx_desc_malformed), + ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_MISSING_ADMIN_INTERRUPT, missing_admin_interrupt), }; int ena_up(struct ena_adapter *adapter); diff --git a/sys/dev/ena/ena_sysctl.c b/sys/dev/ena/ena_sysctl.c index e46e835572fe..c711ecf66d7a 100644 --- a/sys/dev/ena/ena_sysctl.c +++ b/sys/dev/ena/ena_sysctl.c @@ -298,6 +298,10 @@ ena_sysctl_add_stats(struct ena_adapter *adapter) &dev_stats->tx_desc_malformed, "TX descriptors malformed count"); SYSCTL_ADD_COUNTER_U64(ctx, child, OID_AUTO, "rx_desc_malformed", CTLFLAG_RD, &dev_stats->rx_desc_malformed, "RX descriptors malformed count"); + SYSCTL_ADD_COUNTER_U64(ctx, child, OID_AUTO, "missing_admin_interrupt", CTLFLAG_RD, + &dev_stats->missing_admin_interrupt, "Missing admin interrupts count"); + SYSCTL_ADD_COUNTER_U64(ctx, child, OID_AUTO, "admin_to", CTLFLAG_RD, + &dev_stats->admin_to, "Admin queue timeouts count"); SYSCTL_ADD_COUNTER_U64(ctx, child, OID_AUTO, "total_resets", CTLFLAG_RD, &dev_stats->total_resets, "Total resets count");