From nobody Wed Nov 15 23:09:17 2023 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVzPn4DV4z50s1B; Wed, 15 Nov 2023 23:09:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVzPn2l6Zz3NCS; Wed, 15 Nov 2023 23:09:17 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700089757; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=zXKPL95pYFQo4FewN6aZSweP+fJU7caC01DcOaalylo=; b=QZt4Mu35o3WwbfCXkV1S5V+aUGvb7ZvDNAYB2IJ7bN1govLYPtUuzKIyVjIso7H/p+8Frn v5drdCkBR2E1GkvSzVmIJj2dGmGG9u4eVnOLeyAfeWTCBPKpWUoIfCHFqCC0ibr8TPzCm9 hr29sPE3SHh5JuRdutdepTbt1HI9RR81DO4E3HsIwgmwYpwqJsA3ruUAbGs9JmrGV5UDyd +gch8Du+gRVdAui+9YwvH5WqLQ2MNKZ6vSs9TpL4KOAeIBk8L88LjdZTcJ5HBrYVk5PMno qlU+l2dzX4Sx+2N+NMrG0Aqd7HfAr8Zi5FbQKnpHYlr9HpvKuxIUg4mzPHlD7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700089757; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=zXKPL95pYFQo4FewN6aZSweP+fJU7caC01DcOaalylo=; b=NbkmkjQtB84vPp/WkFmAAZ12WPElUQ0vOfF4RjwnGOFhxIZBf/WnZrVVI8cmE2JGjid0N1 gkyY59C0wTviQFtl4QW39L2gb6UufJJM94B6SirVtiXKK8fm4qy4rI9us2OvIuydraJMiO qrYhMwQStLzOFxFGoL4w5C08Tbsh7SFG1Nd/KZttTsqQsOHegxxgmr5nmRtk+S9/nFG/us eKM10gldrNDDK9GPHO67MjObvexEuQ/kEp/AcNtwawraBgErTA/3gT1PD7jg2CHlvA2Y2P xllIkQKpy/AnKWdmUzMKnU4do/hu3KMaHyfQnDU/OAOgM8sTTcKMUQh0reGI9A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700089757; a=rsa-sha256; cv=none; b=wRo9oJSbgQ0U2TUg2BEYjNIEe0Bi3anD1kiYV0sqv6J6WSLBSKK0k/OYffFbDMWZW75+OQ 1ChwamQJxWtLaApBOwEbLa95b/Nkd5U43lPRUXQQ/hivX6Ghv2CuB6mxP0jToKoFkx9sXH UNRqC0vaRf586HtD+vkPw22luSdtvjUf6kg3s5T1vwTmciClPKQ1bV1AKjuSzZbx03QDbb +hiFSS1x8dGfh2R37bAc3XwyGLCRBdR3+HJM12DPT5TXX6gh6SJR1/rTWGh6Jly8+W8fGE foT9KmyWpxUhH31/MT/gkiMO3DhOU2JC/kg1nG01YOQzy+f1ANSEh3F1iTKsfg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4SVzPn1mJ1zbqj; Wed, 15 Nov 2023 23:09:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 3AFN9HRB065102; Wed, 15 Nov 2023 23:09:17 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 3AFN9HuK065099; Wed, 15 Nov 2023 23:09:17 GMT (envelope-from git) Date: Wed, 15 Nov 2023 23:09:17 GMT Message-Id: <202311152309.3AFN9HuK065099@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Konstantin Belousov Subject: git: 7b959396ca6f - main - mlx5: Introduce new destination type TABLE_TYPE List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 7b959396ca6fae5635260131eedb9bc19f2726a3 Auto-Submitted: auto-generated The branch main has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=7b959396ca6fae5635260131eedb9bc19f2726a3 commit 7b959396ca6fae5635260131eedb9bc19f2726a3 Author: Patrisious Haddad AuthorDate: 2023-04-04 06:01:24 +0000 Commit: Konstantin Belousov CommitDate: 2023-11-15 23:08:17 +0000 mlx5: Introduce new destination type TABLE_TYPE This new destination type supports flow transition between different table types, e.g. from NIC_RX to RDMA_RX or from RDMA_TX to NIC_TX. In addition add driver support to be able to query the capability for this new destination type. Signed-off-by: Patrisious Haddad Sponsored by: NVidia networking MFC after: 1 week --- sys/dev/mlx5/device.h | 4 +++ sys/dev/mlx5/mlx5_core/mlx5_main.c | 43 +++++++++++++++++++++++++++++ sys/dev/mlx5/mlx5_ifc.h | 55 ++++++++++++++++++++++++++++++++++++-- 3 files changed, 100 insertions(+), 2 deletions(-) diff --git a/sys/dev/mlx5/device.h b/sys/dev/mlx5/device.h index 11afc68efdf2..4b7cf8686794 100644 --- a/sys/dev/mlx5/device.h +++ b/sys/dev/mlx5/device.h @@ -993,6 +993,7 @@ enum mlx5_cap_type { MLX5_CAP_DEC, MLX5_CAP_TLS, MLX5_CAP_DEV_EVENT = 0x14, + MLX5_CAP_GENERAL_2 = 0x20, /* NUM OF CAP Types */ MLX5_CAP_NUM }; @@ -1031,6 +1032,9 @@ enum mlx5_mcam_feature_groups { #define MLX5_CAP_GEN_MAX(mdev, cap) \ MLX5_GET(cmd_hca_cap, mdev->hca_caps_max[MLX5_CAP_GENERAL], cap) +#define MLX5_CAP_GEN_2(mdev, cap) \ + MLX5_GET(cmd_hca_cap_2, mdev->hca_caps_cur[MLX5_CAP_GENERAL_2], cap) + #define MLX5_CAP_ETH(mdev, cap) \ MLX5_GET(per_protocol_networking_offload_caps,\ mdev->hca_caps_cur[MLX5_CAP_ETHERNET_OFFLOADS], cap) diff --git a/sys/dev/mlx5/mlx5_core/mlx5_main.c b/sys/dev/mlx5/mlx5_core/mlx5_main.c index df38a41a8be0..efaf726e19e5 100644 --- a/sys/dev/mlx5/mlx5_core/mlx5_main.c +++ b/sys/dev/mlx5/mlx5_core/mlx5_main.c @@ -559,6 +559,43 @@ static int handle_hca_cap_atomic(struct mlx5_core_dev *dev) return err; } +static int handle_hca_cap_2(struct mlx5_core_dev *dev) +{ + void *set_ctx; + void *set_hca_cap; + int set_sz = MLX5_ST_SZ_BYTES(set_hca_cap_in); + int err; + + if (MLX5_CAP_GEN_MAX(dev, hca_cap_2)) { + err = mlx5_core_get_caps(dev, MLX5_CAP_GENERAL_2); + if (err) + return err; + } else { + return 0; + } + + /* To be added if sw_vhca support was added */ + /*if (!MLX5_CAP_GEN_2_MAX(dev, sw_vhca_id_valid) || + !(dev->priv.sw_vhca_id > 0)) + return 0;*/ + + set_ctx = kzalloc(set_sz, GFP_KERNEL); + if (!set_ctx) + return -ENOMEM; + + MLX5_SET(set_hca_cap_in, set_ctx, op_mod, + MLX5_CAP_GENERAL_2 << 1); + set_hca_cap = MLX5_ADDR_OF(set_hca_cap_in, set_ctx, capability); + memcpy(set_hca_cap, dev->hca_caps_cur[MLX5_CAP_GENERAL_2], + MLX5_ST_SZ_BYTES(cmd_hca_cap_2)); + //MLX5_SET(cmd_hca_cap_2, set_hca_cap, sw_vhca_id_valid, 1); + + err = set_caps(dev, set_ctx, set_sz); + + kfree(set_ctx); + return err; +} + static int set_hca_ctrl(struct mlx5_core_dev *dev) { struct mlx5_reg_host_endianess he_in; @@ -1139,6 +1176,12 @@ static int mlx5_load_one(struct mlx5_core_dev *dev, struct mlx5_priv *priv, goto reclaim_boot_pages; } + err = handle_hca_cap_2(dev); + if (err) { + mlx5_core_err(dev, "handle_hca_cap_2 failed\n"); + goto reclaim_boot_pages; + } + err = mlx5_satisfy_startup_pages(dev, 0); if (err) { mlx5_core_err(dev, "failed to allocate init pages\n"); diff --git a/sys/dev/mlx5/mlx5_ifc.h b/sys/dev/mlx5/mlx5_ifc.h index 7e76112cbec9..a10cb60dbfdd 100644 --- a/sys/dev/mlx5/mlx5_ifc.h +++ b/sys/dev/mlx5/mlx5_ifc.h @@ -302,6 +302,11 @@ enum { MLX5_CMD_OP_GENERAL_END = 0xd00, }; +enum { + MLX5_FT_NIC_RX_2_NIC_RX_RDMA = BIT(0), + MLX5_FT_NIC_TX_RDMA_2_NIC_TX = BIT(1), +}; + enum { MLX5_ICMD_CMDS_OPCODE_ICMD_OPCODE_QUERY_FW_INFO = 0x8007, MLX5_ICMD_CMDS_OPCODE_ICMD_QUERY_CAPABILITY = 0x8400, @@ -524,7 +529,9 @@ struct mlx5_ifc_dest_format_struct_bits { u8 destination_type[0x8]; u8 destination_id[0x18]; - u8 reserved_0[0x20]; + u8 reserved_0[0x8]; + u8 destination_table_type[0x8]; + u8 reserved_at_1[0x10]; }; struct mlx5_ifc_ipv4_layout_bits { @@ -1144,7 +1151,12 @@ enum { }; struct mlx5_ifc_cmd_hca_cap_bits { - u8 reserved_0[0x80]; + u8 reserved_0[0x20]; + + u8 hca_cap_2[0x1]; + u8 reserved_at_21[0x1f]; + + u8 reserved_at_40[0x40]; u8 log_max_srq_sz[0x8]; u8 log_max_qp_sz[0x8]; @@ -1439,10 +1451,48 @@ struct mlx5_ifc_cmd_hca_cap_bits { u8 reserved_at_7c0[0x40]; }; +struct mlx5_ifc_cmd_hca_cap_2_bits { + u8 reserved_at_0[0x80]; + + u8 migratable[0x1]; + u8 reserved_at_81[0x1f]; + + u8 max_reformat_insert_size[0x8]; + u8 max_reformat_insert_offset[0x8]; + u8 max_reformat_remove_size[0x8]; + u8 max_reformat_remove_offset[0x8]; + + u8 reserved_at_c0[0x8]; + u8 migration_multi_load[0x1]; + u8 migration_tracking_state[0x1]; + u8 reserved_at_ca[0x16]; + + u8 reserved_at_e0[0xc0]; + + u8 flow_table_type_2_type[0x8]; + u8 reserved_at_1a8[0x3]; + u8 log_min_mkey_entity_size[0x5]; + u8 reserved_at_1b0[0x10]; + + u8 reserved_at_1c0[0x60]; + + u8 reserved_at_220[0x1]; + u8 sw_vhca_id_valid[0x1]; + u8 sw_vhca_id[0xe]; + u8 reserved_at_230[0x10]; + + u8 reserved_at_240[0xb]; + u8 ts_cqe_metadata_size2wqe_counter[0x5]; + u8 reserved_at_250[0x10]; + + u8 reserved_at_260[0x5a0]; +}; + enum mlx5_flow_destination_type { MLX5_FLOW_DESTINATION_TYPE_VPORT = 0x0, MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE = 0x1, MLX5_FLOW_DESTINATION_TYPE_TIR = 0x2, + MLX5_FLOW_DESTINATION_TYPE_TABLE_TYPE = 0xA, }; union mlx5_ifc_dest_format_struct_flow_counter_list_auto_bits { @@ -2937,6 +2987,7 @@ struct mlx5_ifc_hca_vport_context_bits { union mlx5_ifc_hca_cap_union_bits { struct mlx5_ifc_cmd_hca_cap_bits cmd_hca_cap; + struct mlx5_ifc_cmd_hca_cap_2_bits cmd_hca_cap_2; struct mlx5_ifc_odp_cap_bits odp_cap; struct mlx5_ifc_atomic_caps_bits atomic_caps; struct mlx5_ifc_roce_cap_bits roce_cap;