From nobody Thu Apr 14 23:01:06 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 21C351B3F408; Thu, 14 Apr 2022 23:01:07 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KfZh30SxWz3lsg; Thu, 14 Apr 2022 23:01:07 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1649977267; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=C8KiMZwTyDa2xz+aAsOZmpMC0VJ4dbsDA/2meLem8p4=; b=fpv7wYZ6o1kgrQPEVZ/+p085hOhvH/JxHA5IC/M9q9pKSLcX4B2fByPvYvkZYLeB+H7nED seDvIV5vGNntUCr7hijakAGf19mZBOgcrCH95CgBBkwVr8gVfdxrwozTbx7ikSnhRmXGcL SYx3gJv4y9j9gFLNXh1KBWdbZ3muxTOmNnTC4i/8c1QJwCRFFsZMjtuOtbQ6hlvkPhpLYI 66UEf6khkBTpRxXiHR1mDE6pKr8EfqcFWA80uT06w4XFRkgUjVvpUVL3bkrd/T8YkyaDcD RYCFrxiuQzG0sh7CB3XWFOAtFtD2yCPtU2dF69QCAQNa5HQSem9WVt4GzxcEaQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E44871840; Thu, 14 Apr 2022 23:01:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 23EN16l1021408; Thu, 14 Apr 2022 23:01:06 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 23EN166U021407; Thu, 14 Apr 2022 23:01:06 GMT (envelope-from git) Date: Thu, 14 Apr 2022 23:01:06 GMT Message-Id: <202204142301.23EN166U021407@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Navdeep Parhar Subject: git: db28d4a0cd1c - main - cxgbe/t4_tom: Support for round-robin selection of offload queues. List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: np X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: db28d4a0cd1c7a8505d6cbb3e8772b794cd55e53 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1649977267; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=C8KiMZwTyDa2xz+aAsOZmpMC0VJ4dbsDA/2meLem8p4=; b=a8+fg/8Pk1paadyXdaG2P1h48/sAGhO8eYY5q0lz0/PdAlpBM8PS3UB9NBJbwizwq05Ott YrW3m/80IZwya61v7G+ac4Cuk0amXOPrmw5sWerKRGgOO5cHpqZRb70zksKC8cv6ZmFp2A OdvLK9ZWxJVm654+tBTpdzuY+jCUL61y4McJBvuIfnFtEE9prM6TKCLYaufuMh16JO1oHM LQD/vgNtWNma8Huidj6SnoZ9Ru/f1IzXkUomUVgg5gdbWL3iqY188WocOKprSORkS+Tobf jBh6wFFxhCNzNBfx7QHX4HxzuiokcKWmSgjI5j979RcrNdbOAwHGmqPKFiK4hg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1649977267; a=rsa-sha256; cv=none; b=DCvLMGyh8qoMVwX68AxpiItGhNZEV3OXkLE2AD002VmLzlA24arwaQfS8w+ugCeLcYQ8aV wg5c5wXrAIG8s5MlSfhAyiU6z+ArMKaNdpoXsSDFh6Z4OyPB+v4Lo+cp04+18eviI5uTw3 Rkfoh6BbvtyeqSR7mCE9v/d1Z4F0UBOAZg643zUDyC0zdESaavwJPzeOufwZ1SLlzXEYzE D/PchrPcNAmuija/gZyGxRM8Vaf/Gj2zbycX2+LUGNVH7vd+irxUGmNEa8qaBgXDB3sfBZ ZcU2K4wYOSPpVbAt0Z1MZ2YoytGtMqwIoVmHPUC2wscHmyf5ATMwzhYS3QxfMw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by np: URL: https://cgit.FreeBSD.org/src/commit/?id=db28d4a0cd1c7a8505d6cbb3e8772b794cd55e53 commit db28d4a0cd1c7a8505d6cbb3e8772b794cd55e53 Author: Navdeep Parhar AuthorDate: 2022-04-14 22:49:58 +0000 Commit: Navdeep Parhar CommitDate: 2022-04-14 22:49:58 +0000 cxgbe/t4_tom: Support for round-robin selection of offload queues. A COP (Connection Offload Policy) rule can now specify that the tx and/or rx queue for a new tid should be selected in a round-robin manner. There is no change in default behavior. Reviewed by: jhb@ MFC after: 1 week Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D34921 --- sys/dev/cxgbe/adapter.h | 2 ++ sys/dev/cxgbe/t4_ioctl.h | 5 +++++ sys/dev/cxgbe/tom/t4_tom.c | 22 ++++++++++++++-------- 3 files changed, 21 insertions(+), 8 deletions(-) diff --git a/sys/dev/cxgbe/adapter.h b/sys/dev/cxgbe/adapter.h index c140d4817f41..637fee6b4a78 100644 --- a/sys/dev/cxgbe/adapter.h +++ b/sys/dev/cxgbe/adapter.h @@ -254,6 +254,8 @@ struct vi_info { struct sysctl_oid *ofld_txq_oid; uint8_t hw_addr[ETHER_ADDR_LEN]; /* factory MAC address, won't change */ + u_int txq_rr; + u_int rxq_rr; }; struct tx_ch_rl_params { diff --git a/sys/dev/cxgbe/t4_ioctl.h b/sys/dev/cxgbe/t4_ioctl.h index f3bb7d8b4aa4..3ef03f7c526c 100644 --- a/sys/dev/cxgbe/t4_ioctl.h +++ b/sys/dev/cxgbe/t4_ioctl.h @@ -376,6 +376,11 @@ enum { OPEN_TYPE_DONTCARE = 'D', }; +enum { + QUEUE_RANDOM = -1, + QUEUE_ROUNDROBIN = -2, +}; + struct offload_settings { int8_t offload; int8_t rx_coalesce; diff --git a/sys/dev/cxgbe/tom/t4_tom.c b/sys/dev/cxgbe/tom/t4_tom.c index 41f55fdbd426..3c1554a8eaf3 100644 --- a/sys/dev/cxgbe/tom/t4_tom.c +++ b/sys/dev/cxgbe/tom/t4_tom.c @@ -1377,17 +1377,23 @@ init_conn_params(struct vi_info *vi , struct offload_settings *s, cp->mtu_idx = find_best_mtu_idx(sc, inc, s); /* Tx queue for this connection. */ - if (s->txq >= 0 && s->txq < vi->nofldtxq) - cp->txq_idx = s->txq; + if (s->txq == QUEUE_RANDOM) + cp->txq_idx = arc4random(); + else if (s->txq == QUEUE_ROUNDROBIN) + cp->txq_idx = atomic_fetchadd_int(&vi->txq_rr, 1); else - cp->txq_idx = arc4random() % vi->nofldtxq; + cp->txq_idx = s->txq; + cp->txq_idx %= vi->nofldtxq; cp->txq_idx += vi->first_ofld_txq; /* Rx queue for this connection. */ - if (s->rxq >= 0 && s->rxq < vi->nofldrxq) - cp->rxq_idx = s->rxq; + if (s->rxq == QUEUE_RANDOM) + cp->rxq_idx = arc4random(); + else if (s->rxq == QUEUE_ROUNDROBIN) + cp->rxq_idx = atomic_fetchadd_int(&vi->rxq_rr, 1); else - cp->rxq_idx = arc4random() % vi->nofldrxq; + cp->rxq_idx = s->rxq; + cp->rxq_idx %= vi->nofldrxq; cp->rxq_idx += vi->first_ofld_rxq; if (SOLISTENING(so)) { @@ -1747,8 +1753,8 @@ lookup_offload_policy(struct adapter *sc, int open_type, struct mbuf *m, .ecn = -1, .ddp = -1, .tls = -1, - .txq = -1, - .rxq = -1, + .txq = QUEUE_RANDOM, + .rxq = QUEUE_RANDOM, .mss = -1, }; static const struct offload_settings disallow_offloading_settings = {