From nobody Fri May 24 15:40:44 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Vm8Q45ssCz5LH4j; Fri, 24 May 2024 15:40:44 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Vm8Q4358hz3whh; Fri, 24 May 2024 15:40:44 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1716565244; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=gCbKW/8cknsYxDNB+HW+mQBNPzZM4rRyn/FPMbRjU00=; b=WOiRPCgRireVM6aubO1/MbJ45oV8ibeddJEKGouxfDvDngJpBeOZxll5XJoqmfZixa+f8v UHbKHaNbAIKIlEcfDu5bEamxqEXxy7p2oYrIl2c2msASb6yfl3gVltkKHFlphfRjJyJPtw FeJrZSG7vdj23ql7UpWVQZifLBngo2oh/Ylq6CogJidJciIvbqUjh20fs32sMv2gMI+Uey 2REYProDQdu5+pC6I6M3ps5WC/MLpTsrXZ63v62M2qJBiUCPRPapkvxEahpu+i1/q8SyoB MVx8EFcIkacpx/ybiCwFyocbQR4IfV+dGKwGPMoRxvC1yBgE1UvCvfhPQtDlmw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1716565244; a=rsa-sha256; cv=none; b=fiVYpoWGhaScrJFR/3nXWu4fcjGYVMpxwj1raB1nTHtZhGBjuH3tlrCpH1D1mwh5H9HoxT tYquPuZwEgbSclDdWHqD/tdu+q9f+ycK6MU/aD5i5WdDdUHLYJdU2JmDBvKnRCYf8OO4aK kDaNb5f4gtakG4wUl7tohfMI0l6OAiEdFKdKobCTgFHy1r2RLOUiHdKIXwHBSZb6iexxh7 pDMJ+jqO/1E3Fw4lzKF0CSRspNh1eHdLtrPYLh9/PpIPbdTjMwAT1GSL0EnKPvd7DK/r3O drqJIqcgmqLgYTFroI11nJxfSkTIcDR1cHOb94UeESk6zz2SfSX6YGgxkx9gsQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1716565244; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=gCbKW/8cknsYxDNB+HW+mQBNPzZM4rRyn/FPMbRjU00=; b=TjzGnawZuBncU0MjamgW8A/q88cAX4Irvv3A6Kb49w830Qy1F8zu3MnCYxIpkxlVHRSS7z m1ywZUaJt6mZSGjBssymXJTY2MLsFn6zNs5F//pmADtB0eXo49aM42tFO6+r1Z0OycTkGM x+iQyOEqOZhNXOyGcxipktSJG24C1QTDYa5ykLuA9m6RkJ5UqjSb9oofsr1r86ra4Qz0fU tRrGI/hErTHvNs8eWJFvOos+BBlFzkL9uOqCtEgqJFRmPmWxQIK4Vow020WFv8ClMvTw/d 77xz1N20W291u6F0SPIysUdkTwnTOQhwa8Z2CfWdIe4N8QW+zBa9YBCyq0OAEg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Vm8Q42f0Yzf8n; Fri, 24 May 2024 15:40:44 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 44OFei8w012326; Fri, 24 May 2024 15:40:44 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 44OFei0I012323; Fri, 24 May 2024 15:40:44 GMT (envelope-from git) Date: Fri, 24 May 2024 15:40:44 GMT Message-Id: <202405241540.44OFei0I012323@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Warner Losh Subject: git: d09ee08f106d - main - nvme: Count number of alginment splits List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: d09ee08f106ddabeb02affc7e9dcfea8c154a57d Auto-Submitted: auto-generated The branch main has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=d09ee08f106ddabeb02affc7e9dcfea8c154a57d commit d09ee08f106ddabeb02affc7e9dcfea8c154a57d Author: Warner Losh AuthorDate: 2024-05-24 14:32:47 +0000 Commit: Warner Losh CommitDate: 2024-05-24 14:32:47 +0000 nvme: Count number of alginment splits When possible, we split up I/Os to NVMe drives that advertise a preferred alignment. Add a counter for this. Sponsored by: Netflix Reviewed by: chuck, mav Differential Revision: https://reviews.freebsd.org/D45311 --- sys/dev/nvme/nvme_ctrlr.c | 5 +++++ sys/dev/nvme/nvme_ns.c | 1 + sys/dev/nvme/nvme_private.h | 4 ++++ sys/dev/nvme/nvme_sysctl.c | 5 +++++ 4 files changed, 15 insertions(+) diff --git a/sys/dev/nvme/nvme_ctrlr.c b/sys/dev/nvme/nvme_ctrlr.c index 00e619bfdc46..d9bd1fce2864 100644 --- a/sys/dev/nvme/nvme_ctrlr.c +++ b/sys/dev/nvme/nvme_ctrlr.c @@ -1442,6 +1442,8 @@ nvme_ctrlr_construct(struct nvme_controller *ctrlr, device_t dev) ctrlr->enable_aborts = 0; TUNABLE_INT_FETCH("hw.nvme.enable_aborts", &ctrlr->enable_aborts); + ctrlr->alignment_splits = counter_u64_alloc(M_WAITOK); + /* Cap transfers by the maximum addressable by page-sized PRP (4KB pages -> 2MB). */ ctrlr->max_xfer_size = MIN(maxphys, (ctrlr->page_size / 8 * ctrlr->page_size)); if (nvme_ctrlr_construct_admin_qpair(ctrlr) != 0) @@ -1560,6 +1562,9 @@ noadminq: ctrlr->resource_id, ctrlr->resource); nores: + if (ctrlr->alignment_splits) + counter_u64_free(ctrlr->alignment_splits); + mtx_destroy(&ctrlr->lock); } diff --git a/sys/dev/nvme/nvme_ns.c b/sys/dev/nvme/nvme_ns.c index e494b8cd857a..201cf9188c1e 100644 --- a/sys/dev/nvme/nvme_ns.c +++ b/sys/dev/nvme/nvme_ns.c @@ -428,6 +428,7 @@ nvme_ns_split_bio(struct nvme_namespace *ns, struct bio *bp, if (child_bios == NULL) return (ENOMEM); + counter_u64_add(ns->ctrlr->alignment_splits, 1); for (i = 0; i < num_bios; i++) { child = child_bios[i]; err = nvme_ns_bio_process(ns, child, nvme_bio_child_done); diff --git a/sys/dev/nvme/nvme_private.h b/sys/dev/nvme/nvme_private.h index 69141add4e48..8415819ca20b 100644 --- a/sys/dev/nvme/nvme_private.h +++ b/sys/dev/nvme/nvme_private.h @@ -32,6 +32,7 @@ #include #include #include +#include #include #include #include @@ -317,6 +318,9 @@ struct nvme_controller { bus_dmamap_t hmb_desc_map; struct nvme_hmb_desc *hmb_desc_vaddr; uint64_t hmb_desc_paddr; + + /* Statistics */ + counter_u64_t alignment_splits; }; #define nvme_mmio_offsetof(reg) \ diff --git a/sys/dev/nvme/nvme_sysctl.c b/sys/dev/nvme/nvme_sysctl.c index d6452a2e5492..447f48e0bdd5 100644 --- a/sys/dev/nvme/nvme_sysctl.c +++ b/sys/dev/nvme/nvme_sysctl.c @@ -30,6 +30,7 @@ #include "opt_nvme.h" #include +#include #include #include @@ -419,4 +420,8 @@ nvme_sysctl_initialize_ctrlr(struct nvme_controller *ctrlr) nvme_sysctl_initialize_queue(&ctrlr->ioq[i], ctrlr_ctx, que_tree); } + + SYSCTL_ADD_COUNTER_U64(ctrlr_ctx, ctrlr_list, OID_AUTO, "alignment_splits", + CTLFLAG_RD, &ctrlr->alignment_splits, + "Number of times we split the I/O alignment for drives with preferred alignment"); }