From nobody Thu Oct 31 16:01:25 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XfTJ61V4Nz5bwZp; Thu, 31 Oct 2024 16:01:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XfTJ56Gjqz4StQ; Thu, 31 Oct 2024 16:01:25 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1730390485; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=b1rBUTh8Ie54t6fyrm/OSwBkVhK8O5zHSt56MXh/y+Q=; b=SEwQLXJgRy9Fx8ZaaOG7GYl5yBm92VPTXnMQEll/pyiiRVKn3NbEzmVVvaZhhOYycPmqVk gf+17FRPO4+O5bnwD0a4DN/nm5OWZPGEMlQWkf5BIec9jgZrdjBQMj8Y3z34xf9zTBffHA Ne0lHDBmLPEubsuMS+2aoj/DE37NoLVhL9hoIFg25/8MvvFZAeS4Zr0lHOBNdgXvBotCAb unAlcUgob+kJFQ/jzsxqgN4l0ZlJRnGhLqHDwFB8j8lSL6JxsNkZS9ObU7TNKAAKjkoulr 6ZWIQvbalmUJSzsinQEFmblg2ewoqNiW4najyYwzbDN0D79Brd/YvAkE+uZxmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1730390485; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=b1rBUTh8Ie54t6fyrm/OSwBkVhK8O5zHSt56MXh/y+Q=; b=K9Pmji8+rzkSNcROnKGWvELpvvi5vczA9gtfXrCDvezGWcHfXRyapMPOExz50LpWI0kh5r 3dfkJWV/P7GRX3nP59Mn55V5sZhcNNpJB1Vb0zt23TmFZ1AAyVDcuxM+pxA+3v1s26J3nC BnxiEf+o9b33EYBQa7/Qvw/4qJ0TnyHuWEE68MCR+7MoziT/UQ/7Vi/Vdzk7hmQtSitUs4 vJP2i0M5kTb/zp6VhDGBOTED8xPhTcwf6gtfB4gfEVaErCwizvfL8PCTy3sGVhx+I5Ax2N TQSjaZKmxicSn8nEyAppBUAEJ+7K9+EPi5ZfSCaJygB+jXa3nODpw055GsLgaw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1730390485; a=rsa-sha256; cv=none; b=X3LwYfwvQUkbhjfeJKh3uMAgmaG+jDAAV/R/dKRn0OZVRA7WM80D/Rv4GVWOUUKC25Aj2F 95nks+ZkfEjCdzRYlibiyU8jI73RqY0GakE/vFcc47/q8aNFZ9L7RAqtHEi8wm+CO+cjnf zjzLUeeziaG7y5JpHjBnvX4axkGmDXEaS3q+cklhkt5M9f7XgEFaVGbQ7CA4Ml+Q6w1+rE ZD/vB/HSkkV7/aCMlX+3a+BC3vA5qP+RnBhzhse4jRmSkKV/rgDwbixmxZdPJOaRjAYKVU kcRgY9223tYzIE4O+AeVWMoJja1ZoJC5cehBSHlJZ0Aid+t9qvs+H/BSgqEkzA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4XfTJ55mkSzjVn; Thu, 31 Oct 2024 16:01:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 49VG1Pbd070055; Thu, 31 Oct 2024 16:01:25 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 49VG1P56070052; Thu, 31 Oct 2024 16:01:25 GMT (envelope-from git) Date: Thu, 31 Oct 2024 16:01:25 GMT Message-Id: <202410311601.49VG1P56070052@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Osama Abboud Subject: git: 89940eed9118 - stable/14 - ena: Improve reset reason statistics List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: osamaabb X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 89940eed91182f4dbb20c14bdbb689fc622dad9b Auto-Submitted: auto-generated The branch stable/14 has been updated by osamaabb: URL: https://cgit.FreeBSD.org/src/commit/?id=89940eed91182f4dbb20c14bdbb689fc622dad9b commit 89940eed91182f4dbb20c14bdbb689fc622dad9b Author: Osama Abboud AuthorDate: 2024-08-07 06:24:19 +0000 Commit: Osama Abboud CommitDate: 2024-10-31 14:54:10 +0000 ena: Improve reset reason statistics The driver uses different reset reasons. Some of them are counted and presented in the driver statistics. There are cases where statistics are counted on a ring level, but these are zeroed after a reset procedure takes place. This commit makes the following changes: 1. Add statistics for the unrepresented reset reasons. 2. Add reset reasons which are counted on a ring level, to be also global for better tracking. Approved by: cperciva (mentor) Sponsored by: Amazon, Inc. (cherry picked from commit 89ce3f6314f6feba0e6626be51832d44df611218) --- sys/dev/ena/ena.c | 2 -- sys/dev/ena/ena.h | 42 ++++++++++++++++++++++++++++++++++++++++++ sys/dev/ena/ena_sysctl.c | 16 ++++++++++++++++ 3 files changed, 58 insertions(+), 2 deletions(-) diff --git a/sys/dev/ena/ena.c b/sys/dev/ena/ena.c index fd92b5046f84..7c86c0594daf 100644 --- a/sys/dev/ena/ena.c +++ b/sys/dev/ena/ena.c @@ -3014,7 +3014,6 @@ check_for_missing_keep_alive(struct ena_adapter *adapter) time = getsbinuptime() - timestamp; if (unlikely(time > adapter->keep_alive_timeout)) { ena_log(adapter->pdev, ERR, "Keep alive watchdog timeout.\n"); - counter_u64_add(adapter->dev_stats.wd_expired, 1); ena_trigger_reset(adapter, ENA_REGS_RESET_KEEP_ALIVE_TO); } } @@ -3026,7 +3025,6 @@ check_for_admin_com_state(struct ena_adapter *adapter) if (unlikely(ena_com_get_admin_running_state(adapter->ena_dev) == false)) { ena_log(adapter->pdev, ERR, "ENA admin queue is not in running state!\n"); - counter_u64_add(adapter->dev_stats.admin_q_pause, 1); ena_trigger_reset(adapter, ENA_REGS_RESET_ADMIN_TO); } } diff --git a/sys/dev/ena/ena.h b/sys/dev/ena/ena.h index be55f63bdb7b..4ac79edd0016 100644 --- a/sys/dev/ena/ena.h +++ b/sys/dev/ena/ena.h @@ -381,6 +381,14 @@ struct ena_stats_dev { counter_u64_t interface_up; counter_u64_t interface_down; counter_u64_t admin_q_pause; + counter_u64_t total_resets; + counter_u64_t os_trigger; + counter_u64_t missing_tx_cmpl; + counter_u64_t bad_rx_req_id; + counter_u64_t bad_tx_req_id; + counter_u64_t bad_rx_desc_num; + counter_u64_t invalid_state; + counter_u64_t missing_intr; }; struct ena_hw_stats { @@ -519,6 +527,29 @@ struct ena_adapter { extern struct sx ena_global_lock; +#define ENA_RESET_STATS_ENTRY(reset_reason, stat) \ + [reset_reason] = { \ + .stat_offset = offsetof(struct ena_stats_dev, stat) / sizeof(u64), \ + .has_counter = true \ +} + +struct ena_reset_stats_offset { + int stat_offset; + bool has_counter; +}; + +static const struct ena_reset_stats_offset resets_to_stats_offset_map[ENA_REGS_RESET_LAST] = { + ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_KEEP_ALIVE_TO, wd_expired), + ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_ADMIN_TO, admin_q_pause), + ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_OS_TRIGGER, os_trigger), + ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_MISS_TX_CMPL, missing_tx_cmpl), + ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_INV_RX_REQ_ID, bad_rx_req_id), + ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_INV_TX_REQ_ID, bad_tx_req_id), + ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_TOO_MANY_RX_DESCS, bad_rx_desc_num), + ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_DRIVER_INVALID_STATE, invalid_state), + ENA_RESET_STATS_ENTRY(ENA_REGS_RESET_MISS_INTERRUPT, missing_intr), +}; + int ena_up(struct ena_adapter *adapter); void ena_down(struct ena_adapter *adapter); int ena_restore_device(struct ena_adapter *adapter); @@ -547,6 +578,17 @@ ena_trigger_reset(struct ena_adapter *adapter, enum ena_regs_reset_reason_types reset_reason) { if (likely(!ENA_FLAG_ISSET(ENA_FLAG_TRIGGER_RESET, adapter))) { + const struct ena_reset_stats_offset *ena_reset_stats_offset = + &resets_to_stats_offset_map[reset_reason]; + + if (ena_reset_stats_offset->has_counter) { + uint64_t *stat_ptr = (uint64_t *)&adapter->dev_stats + + ena_reset_stats_offset->stat_offset; + + counter_u64_add((counter_u64_t)(*stat_ptr), 1); + } + + counter_u64_add(adapter->dev_stats.total_resets, 1); adapter->reset_reason = reset_reason; ENA_FLAG_SET_ATOMIC(ENA_FLAG_TRIGGER_RESET, adapter); } diff --git a/sys/dev/ena/ena_sysctl.c b/sys/dev/ena/ena_sysctl.c index a94bcbccdc98..6eafe2a8c052 100644 --- a/sys/dev/ena/ena_sysctl.c +++ b/sys/dev/ena/ena_sysctl.c @@ -280,6 +280,22 @@ ena_sysctl_add_stats(struct ena_adapter *adapter) "Network interface down count"); SYSCTL_ADD_COUNTER_U64(ctx, child, OID_AUTO, "admin_q_pause", CTLFLAG_RD, &dev_stats->admin_q_pause, "Admin queue pauses"); + SYSCTL_ADD_COUNTER_U64(ctx, child, OID_AUTO, "os_trigger", CTLFLAG_RD, + &dev_stats->os_trigger, "OS trigger count"); + SYSCTL_ADD_COUNTER_U64(ctx, child, OID_AUTO, "missing_tx_cmpl", CTLFLAG_RD, + &dev_stats->missing_tx_cmpl, "Missing TX completions resets count"); + SYSCTL_ADD_COUNTER_U64(ctx, child, OID_AUTO, "bad_rx_req_id", CTLFLAG_RD, + &dev_stats->bad_rx_req_id, "Bad RX req id count"); + SYSCTL_ADD_COUNTER_U64(ctx, child, OID_AUTO, "bad_tx_req_id", CTLFLAG_RD, + &dev_stats->bad_tx_req_id, "Bad TX req id count"); + SYSCTL_ADD_COUNTER_U64(ctx, child, OID_AUTO, "bad_rx_desc_num", CTLFLAG_RD, + &dev_stats->bad_rx_desc_num, "Bad RX descriptors number count"); + SYSCTL_ADD_COUNTER_U64(ctx, child, OID_AUTO, "invalid_state", CTLFLAG_RD, + &dev_stats->invalid_state, "Driver invalid state count"); + SYSCTL_ADD_COUNTER_U64(ctx, child, OID_AUTO, "missing_intr", CTLFLAG_RD, + &dev_stats->missing_intr, "Missing interrupt count"); + SYSCTL_ADD_COUNTER_U64(ctx, child, OID_AUTO, "total_resets", CTLFLAG_RD, + &dev_stats->total_resets, "Total resets count"); for (i = 0; i < adapter->num_io_queues; ++i, ++tx_ring, ++rx_ring) { snprintf(namebuf, QUEUE_NAME_LEN, "queue%d", i);