From nobody Thu Sep 14 12:12:04 2023 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Rmbm43Gb4z4t4Pm; Thu, 14 Sep 2023 12:12:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Rmbm42qyqz3SPL; Thu, 14 Sep 2023 12:12:04 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1694693524; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=iswkvVUNN6fbkq8ilkWfgWJuyr2HT+QhntWx5gdjLk8=; b=P6JMXs41D734vBXFoqLbLzkze7z9V9FUJFmGOV/aonwD1uBGWMOEk6njCUcfmCl7O0jEOX 0N97sEI1D4CMO2TzVbpBslBkdwiqMUIAC340xiSi8GxAnFYcFrOl27+pAhNLVqPakVAAPs Tw66HlnAolrzkJoSiMyuqVWDq0UgLDQMaadsrft+V8da2UBh5lFqIYtjSacrbW7RS5E5F9 r8AXQdBG16bmm/sm2lk41l0sNlAMXZCQJxlPaWCiyuhsNpKMEYbaYgEXj3xHseuUoihkH+ I/udqphw4nlBVZBdfZJsIbJBUbe6jPIDzIG6+XjuymInDEWoeovcftLqm/0otQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1694693524; a=rsa-sha256; cv=none; b=oKCyMsZK1j3xpGc/aRqkWeBC+Of/Djl85lLhTvUg/PDDcamI6UTCYWp0cYGF9iDKRGMvBw gwikg4MaeMCxkPt/GO+AU7+T4feQyHlomupSiavIS4Es1NyU7GcVUuiNrvTDEhfG15vCOC wWZm0rWQ3NSnEgnFdZPWWHHTXT2M39mjfCn9N7Yu0lwa5phEXN3KXmSTP5Ufgx/OsbsFsB EQ4/NtRDnbnOCg7+k1STgtaoaoUpIxWdnvR4yRYOuHzC7iLgJFI+2j6wOcamNcKCDXXQJc l92bOYRjBAdQ+8daLGZ2QnHInHe5UOImdYEaEx/CEdG6GOV4EthMosQt5Hmd/A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1694693524; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=iswkvVUNN6fbkq8ilkWfgWJuyr2HT+QhntWx5gdjLk8=; b=Lw4K0niCL+PFBwibON74pVXmkLm4QNZHuxwvKZ9D98SC8FfeUFENjw4pC5nBhvxGjsvcrb No11+bCC4Ir9EG17tw5Xb+Q7CAbU7KuZo/1dg2TtzJuY+d1P8gdgsI3YfdNPT6uK8wfa/7 GrFj83R8SgAaHboaVOseULpkTzAnfeJ8GCmMcIFfi2I5m14tff2bcVuGh7sTgNA6TlsbEE BlDtXGqfcAJWlgjrnNu7kAoh5KdgiCF+Otnd/UhKRLTPaiq/IdfLQaL+Nne5J+kmjXUjrG JhqK195w9Vi7vH1jLeNSfd/jGzdMWNtpo+UT9FeWxda23LMCdWvroByvMiL1+g== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Rmbm41vzTzx26; Thu, 14 Sep 2023 12:12:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 38ECC4eH009241; Thu, 14 Sep 2023 12:12:04 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 38ECC4Am009223; Thu, 14 Sep 2023 12:12:04 GMT (envelope-from git) Date: Thu, 14 Sep 2023 12:12:04 GMT Message-Id: <202309141212.38ECC4Am009223@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Wei Hu Subject: git: b167e449c8db - main - mana: add lro and tso stat counters List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: whu X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: b167e449c8db01f082691503fb5c1255ad5750eb Auto-Submitted: auto-generated The branch main has been updated by whu: URL: https://cgit.FreeBSD.org/src/commit/?id=b167e449c8db01f082691503fb5c1255ad5750eb commit b167e449c8db01f082691503fb5c1255ad5750eb Author: Wei Hu AuthorDate: 2023-09-14 11:56:20 +0000 Commit: Wei Hu CommitDate: 2023-09-14 12:10:49 +0000 mana: add lro and tso stat counters Add a few stat counters for tso and lro. MFC after: 3 days Sponsored by: Microsoft --- sys/dev/mana/mana.h | 5 ++ sys/dev/mana/mana_en.c | 12 ++++ sys/dev/mana/mana_sysctl.c | 136 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 153 insertions(+) diff --git a/sys/dev/mana/mana.h b/sys/dev/mana/mana.h index 9a22ea87baa3..ab394f3203ad 100644 --- a/sys/dev/mana/mana.h +++ b/sys/dev/mana/mana.h @@ -170,6 +170,9 @@ struct mana_txq { struct mtx txq_mtx; char txq_mtx_name[16]; + uint64_t tso_pkts; + uint64_t tso_bytes; + struct task enqueue_task; struct taskqueue *enqueue_tq; @@ -423,6 +426,8 @@ struct mana_rxq { uint32_t buf_index; + uint64_t lro_tried; + uint64_t lro_failed; struct mana_stats stats; /* MUST BE THE LAST MEMBER: diff --git a/sys/dev/mana/mana_en.c b/sys/dev/mana/mana_en.c index 064b28fa94a4..812a42e51dfc 100644 --- a/sys/dev/mana/mana_en.c +++ b/sys/dev/mana/mana_en.c @@ -501,6 +501,7 @@ mana_xmit(struct mana_txq *txq) struct gdma_queue *gdma_sq; struct mana_cq *cq; int err, len; + bool is_tso; gdma_sq = txq->gdma_sq; cq = &apc->tx_qp[txq->idx].tx_cq; @@ -578,7 +579,10 @@ mana_xmit(struct mana_txq *txq) pkg.wqe_req.flags = 0; pkg.wqe_req.client_data_unit = 0; + is_tso = false; if (mbuf->m_pkthdr.csum_flags & CSUM_TSO) { + is_tso = true; + if (MANA_L3_PROTO(mbuf) == ETHERTYPE_IP) pkg.tx_oob.s_oob.is_outer_ipv4 = 1; else @@ -641,6 +645,11 @@ mana_xmit(struct mana_txq *txq) packets++; bytes += len; + + if (is_tso) { + txq->tso_pkts++; + txq->tso_bytes += len; + } } counter_enter(); @@ -1697,9 +1706,12 @@ mana_rx_mbuf(struct mbuf *mbuf, struct mana_rxcomp_oob *cqe, do_if_input = true; if ((if_getcapenable(ndev) & IFCAP_LRO) && do_lro) { + rxq->lro_tried++; if (rxq->lro.lro_cnt != 0 && tcp_lro_rx(&rxq->lro, mbuf, 0) == 0) do_if_input = false; + else + rxq->lro_failed++; } if (do_if_input) { if_input(ndev, mbuf); diff --git a/sys/dev/mana/mana_sysctl.c b/sys/dev/mana/mana_sysctl.c index 79a4867a0abf..d03ebe44ab31 100644 --- a/sys/dev/mana/mana_sysctl.c +++ b/sys/dev/mana/mana_sysctl.c @@ -46,6 +46,96 @@ SYSCTL_INT(_hw_mana, OID_AUTO, log_level, CTLFLAG_RWTUN, SYSCTL_CONST_STRING(_hw_mana, OID_AUTO, driver_version, CTLFLAG_RD, DRV_MODULE_VERSION, "MANA driver version"); +static int +mana_sysctl_rx_stat_agg_u64(SYSCTL_HANDLER_ARGS) +{ + struct mana_port_context *apc = arg1; + int offset = arg2, i, err; + struct mana_rxq *rxq; + uint64_t stat; + + stat = 0; + for (i = 0; i < apc->num_queues; i++) { + rxq = apc->rxqs[i]; + stat += *((uint64_t *)((uint8_t *)rxq + offset)); + } + + err = sysctl_handle_64(oidp, &stat, 0, req); + if (err || req->newptr == NULL) + return err; + + for (i = 0; i < apc->num_queues; i++) { + rxq = apc->rxqs[i]; + *((uint64_t *)((uint8_t *)rxq + offset)) = 0; + } + return 0; +} + +static int +mana_sysctl_rx_stat_u16(SYSCTL_HANDLER_ARGS) +{ + struct mana_port_context *apc = arg1; + int offset = arg2, err; + struct mana_rxq *rxq; + uint64_t stat; + uint16_t val; + + rxq = apc->rxqs[0]; + val = *((uint16_t *)((uint8_t *)rxq + offset)); + stat = val; + + err = sysctl_handle_64(oidp, &stat, 0, req); + if (err || req->newptr == NULL) + return err; + else + return 0; +} + +static int +mana_sysctl_rx_stat_u32(SYSCTL_HANDLER_ARGS) +{ + struct mana_port_context *apc = arg1; + int offset = arg2, err; + struct mana_rxq *rxq; + uint64_t stat; + uint32_t val; + + rxq = apc->rxqs[0]; + val = *((uint32_t *)((uint8_t *)rxq + offset)); + stat = val; + + err = sysctl_handle_64(oidp, &stat, 0, req); + if (err || req->newptr == NULL) + return err; + else + return 0; +} + +static int +mana_sysctl_tx_stat_agg_u64(SYSCTL_HANDLER_ARGS) +{ + struct mana_port_context *apc = arg1; + int offset = arg2, i, err; + struct mana_txq *txq; + uint64_t stat; + + stat = 0; + for (i = 0; i < apc->num_queues; i++) { + txq = &apc->tx_qp[i].txq; + stat += *((uint64_t *)((uint8_t *)txq + offset)); + } + + err = sysctl_handle_64(oidp, &stat, 0, req); + if (err || req->newptr == NULL) + return err; + + for (i = 0; i < apc->num_queues; i++) { + txq = &apc->tx_qp[i].txq; + *((uint64_t *)((uint8_t *)txq + offset)) = 0; + } + return 0; +} + void mana_sysctl_add_port(struct mana_port_context *apc) { @@ -99,6 +189,52 @@ mana_sysctl_add_port(struct mana_port_context *apc) CTLFLAG_RD, &port_stats->rx_drops, "Receive packet drops"); SYSCTL_ADD_COUNTER_U64(ctx, stats_list, OID_AUTO, "tx_drops", CTLFLAG_RD, &port_stats->tx_drops, "Transmit packet drops"); + + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "rx_lro_queued", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_MPSAFE | CTLFLAG_STATS, apc, + __offsetof(struct mana_rxq, lro.lro_queued), + mana_sysctl_rx_stat_agg_u64, "LU", "LRO queued"); + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "rx_lro_flushed", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_MPSAFE | CTLFLAG_STATS, apc, + __offsetof(struct mana_rxq, lro.lro_flushed), + mana_sysctl_rx_stat_agg_u64, "LU", "LRO flushed"); + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "rx_lro_bad_csum", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_MPSAFE | CTLFLAG_STATS, apc, + __offsetof(struct mana_rxq, lro.lro_bad_csum), + mana_sysctl_rx_stat_agg_u64, "LU", "LRO bad checksum"); + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "rx_lro_tried", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_STATS, apc, + __offsetof(struct mana_rxq, lro_tried), + mana_sysctl_rx_stat_agg_u64, "LU", "LRO tried"); + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "rx_lro_failed", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_STATS, apc, + __offsetof(struct mana_rxq, lro_failed), + mana_sysctl_rx_stat_agg_u64, "LU", "LRO failed"); + + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "lro_ackcnt_lim", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_STATS, apc, + __offsetof(struct mana_rxq, lro.lro_ackcnt_lim), + mana_sysctl_rx_stat_u16, + "LU", "Max # of ACKs to be aggregated by LRO"); + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "lro_length_lim", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_STATS, apc, + __offsetof(struct mana_rxq, lro.lro_length_lim), + mana_sysctl_rx_stat_u32, + "LU", "Max len of aggregated data in byte by LRO"); + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "lro_cnt", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_STATS, apc, + __offsetof(struct mana_rxq, lro.lro_cnt), + mana_sysctl_rx_stat_u32, + "LU", "Max # or LRO packet count"); + + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "tx_tso_packets", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_STATS, apc, + __offsetof(struct mana_txq, tso_pkts), + mana_sysctl_tx_stat_agg_u64, "LU", "TSO packets"); + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "tx_tso_bytes", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_STATS, apc, + __offsetof(struct mana_txq, tso_bytes), + mana_sysctl_tx_stat_agg_u64, "LU", "TSO bytes"); } void