From nobody Tue Jul 26 19:31:05 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Lsn895vHbz4WwGS; Tue, 26 Jul 2022 19:31:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Lsn895Rm9z43KL; Tue, 26 Jul 2022 19:31:05 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1658863865; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=cXQE7grJJDNBrnIdRsxNbx2H7mhdy6EKKKEneaglBqI=; b=nwOjii1tkmlHs3YnCJ5Uopz9afK3z+GJ0Eqniznd04uj9nlVx8DquGW1NMi2yeOWsKo5Qa 87Tfok8E+Rs7R1BNwBqVgZcSinyY4O2+hukLGy+fXq/Z44AMZ+NT0/sG60I5MDFetXVgum zU/gRGWLSLyReXvLDpC7fTPOvG8gffcC7BQLQLG3TO1lvSA/od8qPOi7R/X6L+2VvMjXB6 yj5prSgytMZuHRmuMqywANFrAWm1Rv4ytnCvjNN63K9N8JRGkaPCDRAtR1/mQH2pNxOf8i ky5P6lRSrWWOfrsXZohyTYwKXnE8MtZ5lRH/MmqryYyWmSXe4ktAX8LQaMBBUw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Lsn894RybzpMl; Tue, 26 Jul 2022 19:31:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 26QJV5ug029916; Tue, 26 Jul 2022 19:31:05 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 26QJV5Sn029915; Tue, 26 Jul 2022 19:31:05 GMT (envelope-from git) Date: Tue, 26 Jul 2022 19:31:05 GMT Message-Id: <202207261931.26QJV5Sn029915@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Marcin Wojtas Subject: git: e8253e47e1dc - stable/13 - ena: Fix LLQ descriptor reconfiguration List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mw X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: e8253e47e1dcc29d07c622e4a028df4780807aa6 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1658863865; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=cXQE7grJJDNBrnIdRsxNbx2H7mhdy6EKKKEneaglBqI=; b=xAAD4K8ebM9Ym2Y8ACbGrnkKxSR1vvpReB3u4x5S84AZvS0qE9sLAH6QTLhdy+YVzYlcks xktSabEzpryOLApinEgTDIWsGOz11jGtqM1QvQLibX4GcSEhTHoi5LMvz450HnVeo4xBhI 5YXH78/ZVcIUKSv4CR0gVbXQIbiNf/cr9eREn3ZuolP6vNSynKkJIr6+WnIhPzeKeBnLun u9CGZr867XdQdJkVcR3PHW38iMHWVii60acD0OEy6X9XOAXKlf/iBIbVmIYggLazbim+09 qYAAbB7QKb9BsxAImUSqD1bmJ4zKTIkNa8GfRyG4YOPNXoMdMRStW/TIVV8tpA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1658863865; a=rsa-sha256; cv=none; b=d3LiImepfVaJTqpV5G0vNH8xibeGcgZRUCyAsGId1UzmD3zkyghRkPS0RMCh6kmMqc4xoL wvGi299/xNoWidZ2fgkixOwv5QMtOF0HH7igkNHvJ6z6dQBu5h4h9vho4kTHkFhBqRT8fE 81XsZ7QguHDi4C1WdeejlNf2t7q833jqci0w98T9F4KrKz0w86KaV5yfcYCBZ0pgYqX1Hw dU0SzypUHySSGOtY944+gI0f+tpQ6BBQLwc7+vKXm7yyeAjqT+Xj55wrBLkvO7viOS1svd MRL1bWvPBOpLLLAzTDBaBylQ0E+WlOXo+bGRTejzW4IWlsaicxCH41jkVqZMTQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by mw: URL: https://cgit.FreeBSD.org/src/commit/?id=e8253e47e1dcc29d07c622e4a028df4780807aa6 commit e8253e47e1dcc29d07c622e4a028df4780807aa6 Author: Michal Krawczyk AuthorDate: 2022-07-05 10:59:25 +0000 Commit: Marcin Wojtas CommitDate: 2022-07-26 19:30:17 +0000 ena: Fix LLQ descriptor reconfiguration After the device reset, the LLQ configuration descriptor wasn't passed to the hardware. On a 6-generation AWS instances (like C6gn), it is required to pass the LLQ descriptor after the device reset, otherwise the hardware will be missing the LLQ configuration resulting in performance degradation. This patch reconfigures the LLQ each time the ena_device_init() is called. This means that the LLQ descriptor will be passed during the initial configuration and after a reset. The ena_map_llq_mem_bar() function call was moved before the ena_device_init() call, to make sure that the mem bar is available. Obtained from: Semihalf MFC after: 2 weeks Sponsored by: Amazon, Inc. (cherry picked from commit 3324e304c14d121dc04338a38c7f3277cbfba58b) --- sys/dev/ena/ena.c | 35 ++++++++++++++++------------------- 1 file changed, 16 insertions(+), 19 deletions(-) diff --git a/sys/dev/ena/ena.c b/sys/dev/ena/ena.c index ad4fa357e595..86ac5941faad 100644 --- a/sys/dev/ena/ena.c +++ b/sys/dev/ena/ena.c @@ -2579,8 +2579,7 @@ ena_map_llq_mem_bar(device_t pdev, struct ena_com_dev *ena_dev) RF_ACTIVE); if (unlikely(adapter->memory == NULL)) { ena_log(pdev, WARN, - "unable to allocate LLQ bar resource. Fallback to host mode policy.\n"); - ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_HOST; + "Unable to allocate LLQ bar resource. LLQ mode won't be used.\n"); return (0); } @@ -2765,6 +2764,7 @@ static int ena_device_init(struct ena_adapter *adapter, device_t pdev, struct ena_com_dev_get_features_ctx *get_feat_ctx, int *wd_active) { + struct ena_llq_configurations llq_config; struct ena_com_dev *ena_dev = adapter->ena_dev; bool readless_supported; uint32_t aenq_groups; @@ -2844,6 +2844,15 @@ ena_device_init(struct ena_adapter *adapter, device_t pdev, *wd_active = !!(aenq_groups & BIT(ENA_ADMIN_KEEP_ALIVE)); + set_default_llq_configurations(&llq_config, &get_feat_ctx->llq); + + rc = ena_set_queues_placement_policy(pdev, ena_dev, &get_feat_ctx->llq, + &llq_config); + if (unlikely(rc != 0)) { + ena_log(pdev, ERR, "Failed to set placement policy\n"); + goto err_admin_init; + } + return (0); err_admin_init: @@ -3487,7 +3496,6 @@ static int ena_attach(device_t pdev) { struct ena_com_dev_get_features_ctx get_feat_ctx; - struct ena_llq_configurations llq_config; struct ena_calc_queue_size_ctx calc_queue_ctx = { 0 }; static int version_printed; struct ena_adapter *adapter; @@ -3560,7 +3568,11 @@ ena_attach(device_t pdev) goto err_bus_free; } - ena_dev->tx_mem_queue_type = ENA_ADMIN_PLACEMENT_POLICY_HOST; + rc = ena_map_llq_mem_bar(pdev, ena_dev); + if (unlikely(rc != 0)) { + ena_log(pdev, ERR, "Failed to map ENA mem bar"); + goto err_bus_free; + } /* Initially clear all the flags */ ENA_FLAG_ZERO(adapter); @@ -3573,21 +3585,6 @@ ena_attach(device_t pdev) goto err_bus_free; } - set_default_llq_configurations(&llq_config, &get_feat_ctx.llq); - - rc = ena_map_llq_mem_bar(pdev, ena_dev); - if (unlikely(rc != 0)) { - ena_log(pdev, ERR, "failed to map ENA mem bar"); - goto err_com_free; - } - - rc = ena_set_queues_placement_policy(pdev, ena_dev, &get_feat_ctx.llq, - &llq_config); - if (unlikely(rc != 0)) { - ena_log(pdev, ERR, "failed to set placement policy\n"); - goto err_com_free; - } - if (ena_dev->tx_mem_queue_type == ENA_ADMIN_PLACEMENT_POLICY_DEV) adapter->disable_meta_caching = !!( get_feat_ctx.llq.accel_mode.u.get.supported_flags &