From nobody Mon Jun 17 02:11:37 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4W2YKP6mxYz5N291; Mon, 17 Jun 2024 02:11:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4W2YKP53pQz4f2p; Mon, 17 Jun 2024 02:11:37 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1718590297; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=10Ht+DwlXOW+xVCKnl5581GMypFC3QYZkyR0NXGyeo0=; b=bJQc73W2LuDwF7Ajfyfxfc7EaJjbdaKtOv34cbXbZGzGCIBwy/QIkAU9odowb9iFoMKQau C/cOmZuVEnKT8hW91AwTZ48o6/4Xkr/zSi2J9+rABg+GCFM4sG0SeP6o05DS0S0gW4V+zI 9lB21isXTQ8BcStIOiS5rt4vuOaqrkyxm1fF6lzWRBDjUphrM/LowOpQGcNpmQoZoGTzdE JDuX3JJpktKyXUs6HNj3ZJusOfqHr9HbYxxlwU7Vz7BIqDr2LNmH27kxRFOqccf1OJANkJ 0PkXtvgMvkeGi87AFAGdpSV4ew8lqSULrX7Is8+uh/TViDEznahxTVuLs10/YA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1718590297; a=rsa-sha256; cv=none; b=ipRsz+Ntor+UsbC2bu830Pg0/a18H574R0zhazsJRz6mRI6K/MIQLRCYG/4zYoYbaYdx+B PArec7A1aGR0ezgsLkUJjB260IqhW/VUTqhOmW/nnlenEbNcEhzrubKFHA+2jJxejlRX9v fLiMxuyiSi379lvNXdoDHuxBos+GHYess+Pdy+8+7wB31q71407rYHIe/kcH8q0bs8CPSC gFCLSDaIwxCVijXFNIoBgmAAa7pQsJhz8kXVKM4tTb3Gt7K6bjrwD4sNCchhjpP5rJ/ugY pX852+iIkEDbvLGpknTC0HcwVNVuV8jT2gMH35H9YF49T5TvPVxAIuZ8Qm1DZA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1718590297; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=10Ht+DwlXOW+xVCKnl5581GMypFC3QYZkyR0NXGyeo0=; b=gcIIjloRnNJUH5CI79+gJWwDp8i3CqHWb3eqL6rwAzwCq6MAC7DP8LG0lahvVoLhD/FNFR vAK9Z22UKo/VrH9s8gbdFHhduBJpNQT+/dcbcfz5btJwyPwC2bpBVZFNOMWKRdGNqhPOI1 3vzGz8WURM+wqsSu7XhkJS8FQeykQbhv8xXwKsRBBEM8NUrsDAOJRgyw+QS585mmmnl8GI B5QCpYAgAfuLT2wp+/XOpVN1aQxJwWl8q3FDch8Z9FzHztwO78KCCxXx+Cbd2876wH3yy6 7JsyggM/bNMdIInlt2nRuCILpRBfLmoGsiIeKvbljUKJZSPxgRZYI+o6D7ALkA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4W2YKP4frLz1BvK; Mon, 17 Jun 2024 02:11:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 45H2BbGp046708; Mon, 17 Jun 2024 02:11:37 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 45H2BboF046705; Mon, 17 Jun 2024 02:11:37 GMT (envelope-from git) Date: Mon, 17 Jun 2024 02:11:37 GMT Message-Id: <202406170211.45H2BboF046705@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Navdeep Parhar Subject: git: ba95b4aea789 - main - cxgbe(4): New knob to limit driver to the specified types of doorbells. List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: np X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: ba95b4aea78909bca972239afcbd51538c4b37c1 Auto-Submitted: auto-generated The branch main has been updated by np: URL: https://cgit.FreeBSD.org/src/commit/?id=ba95b4aea78909bca972239afcbd51538c4b37c1 commit ba95b4aea78909bca972239afcbd51538c4b37c1 Author: Navdeep Parhar AuthorDate: 2024-06-13 17:22:03 +0000 Commit: Navdeep Parhar CommitDate: 2024-06-17 01:54:02 +0000 cxgbe(4): New knob to limit driver to the specified types of doorbells. hw.cxgbe.doorbells_allowed="0xf" The adapter's doorbells bitmap is clipped to the value specified in the tunable, which is meant for debug and workarounds only. There is no change in default behavior. MFC after: 1 week Sponsored by: Chelsio Communications --- sys/dev/cxgbe/adapter.h | 1 + sys/dev/cxgbe/t4_main.c | 21 +++++++++++++++++++++ sys/dev/cxgbe/t4_vf.c | 4 ++++ 3 files changed, 26 insertions(+) diff --git a/sys/dev/cxgbe/adapter.h b/sys/dev/cxgbe/adapter.h index 1a61834c5a40..9ab672ba6442 100644 --- a/sys/dev/cxgbe/adapter.h +++ b/sys/dev/cxgbe/adapter.h @@ -1364,6 +1364,7 @@ void t4_add_adapter(struct adapter *); int t4_detach_common(device_t); int t4_map_bars_0_and_4(struct adapter *); int t4_map_bar_2(struct adapter *); +int t4_adj_doorbells(struct adapter *); int t4_setup_intr_handlers(struct adapter *); void t4_sysctls(struct adapter *); int begin_synchronized_op(struct adapter *, struct vi_info *, int, char *); diff --git a/sys/dev/cxgbe/t4_main.c b/sys/dev/cxgbe/t4_main.c index ae18efb25f15..96407e7ebaba 100644 --- a/sys/dev/cxgbe/t4_main.c +++ b/sys/dev/cxgbe/t4_main.c @@ -604,6 +604,11 @@ static int t5_write_combine = 0; SYSCTL_INT(_hw_cxl, OID_AUTO, write_combine, CTLFLAG_RDTUN, &t5_write_combine, 0, "Use WC instead of UC for BAR2"); +/* From t4_sysctls: doorbells = {"\20\1UDB\2WCWR\3UDBWC\4KDB"} */ +static int t4_doorbells_allowed = 0xf; +SYSCTL_INT(_hw_cxgbe, OID_AUTO, doorbells_allowed, CTLFLAG_RDTUN, + &t4_doorbells_allowed, 0, "Limit tx queues to these doorbells"); + static int t4_num_vis = 1; SYSCTL_INT(_hw_cxgbe, OID_AUTO, num_vis, CTLFLAG_RDTUN, &t4_num_vis, 0, "Number of VIs per port"); @@ -1347,6 +1352,10 @@ t4_attach(device_t dev) if (rc != 0) goto done; /* error message displayed already */ + rc = t4_adj_doorbells(sc); + if (rc != 0) + goto done; /* error message displayed already */ + rc = t4_create_dma_tag(sc); if (rc != 0) goto done; /* error message displayed already */ @@ -3760,6 +3769,18 @@ t4_map_bar_2(struct adapter *sc) return (0); } +int +t4_adj_doorbells(struct adapter *sc) +{ + if ((sc->doorbells & t4_doorbells_allowed) != 0) { + sc->doorbells &= t4_doorbells_allowed; + return (0); + } + CH_ERR(sc, "No usable doorbell (available = 0x%x, allowed = 0x%x).\n", + sc->doorbells, t4_doorbells_allowed); + return (EINVAL); +} + struct memwin_init { uint32_t base; uint32_t aperture; diff --git a/sys/dev/cxgbe/t4_vf.c b/sys/dev/cxgbe/t4_vf.c index a4a611ed645c..24c80aaa7b2d 100644 --- a/sys/dev/cxgbe/t4_vf.c +++ b/sys/dev/cxgbe/t4_vf.c @@ -589,6 +589,10 @@ t4vf_attach(device_t dev) if (rc != 0) goto done; /* error message displayed already */ + rc = t4_adj_doorbells(sc); + if (rc != 0) + goto done; /* error message displayed already */ + rc = t4_create_dma_tag(sc); if (rc != 0) goto done; /* error message displayed already */