From nobody Sat Jul 20 02:59:06 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WQrpy5gVDz5Qxtt; Sat, 20 Jul 2024 02:59:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WQrpy3Zhbz43BH; Sat, 20 Jul 2024 02:59:06 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721444346; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=YssqplutNgRt72tXtet9QfYu7Ko0FRSgOIXV9HoYa9M=; b=sbtRN/QigeqTnmJuX6YulRpsVA37EG5tjka2pk/WinkPui0Z1orIfXH0QH5Oj+mF0iIb72 eqymNl31mamt13gKWbFxMkjT5RhMvoOr06OkF0/8fP6YOxc8b2mr9VidAcaZLgrn8lbVVU yyD/CT8NuCsJ8Bg79hZAumbYbgjjvoLqlAX0gzT8689ve6AW/rfhCtpOLL6OBqlOpLpNtd nsa2BFzeseZrx7dgAXgjvIPlcKbKoAhBcfh4/mCnde2Zt2TprFsRJlV8ToCbpuPlmIoqOR CaY+s0eJIU+eXfJyj3gnkpV3m7O/N92YvUzjhE0knHUXYEJYBk7F1EkOPnGWZg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1721444346; a=rsa-sha256; cv=none; b=sHvdtCNo9WsCs5nPpNeTkOxFXMna5cCmzKMzh+7HrKuEKMiQXM8p53ryYctzdfqNC5bYpX kmA1zrJpF83c1vLG+xufz0jQkCGvElDSK5Rbdd6DiqUiXI6+IY5qJ7B9idE8//4J6pjs6m csUZID1nDM+iLKoufOw26hFmoLjf/s3aHNbtfs+WYby9f266CAm7leftoQXLQyaUI3r43i 2IuJczXqscT0IHDo7wjf3l5KfeoiaDcxWXxbZxMy8ktIRuKumSJXZz9nK3RQnNbS01Ij2X tN124CZFr+VtiDugS0z0aEAng6qpAI/Bbt5Vf9Qbyad49RNjR6IwdUnZAK29Hw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721444346; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=YssqplutNgRt72tXtet9QfYu7Ko0FRSgOIXV9HoYa9M=; b=J1KPEpduiRkP4oex4xFcgcYBDTKzpAEbCQiBnqiotKJvvIOQlbg2FIEPnww85uIyvrlq4C vv0u3dBx7Xskv8sbrLM+pQiP6O1S0T6aBBl/rZHmF9MnLkLRgt0IMGOE9fb7F2qZgAMcrP LZAcI3gOhHq648c5Wu8LTPl/pGWr1K+M04H9NWzBXmQFKL5GAUDrXs0SCfEwhfklDyRP5H ab9QohegGC/7JJ2CzTpklnyE6JQOxEXjL/prcQFYM3RCaEpjs7FaiC0Dhgb209JTlAX8+Y 6G1QVbj8UQ82bB1SpSkJllNAkmyB9b7c1JvB6aqk1TeDQ9FkC3WcAheK/Ai+2A== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WQrpy33z8zyVQ; Sat, 20 Jul 2024 02:59:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 46K2x6g9024398; Sat, 20 Jul 2024 02:59:06 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 46K2x6Vt024395; Sat, 20 Jul 2024 02:59:06 GMT (envelope-from git) Date: Sat, 20 Jul 2024 02:59:06 GMT Message-Id: <202407200259.46K2x6Vt024395@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Warner Losh Subject: git: e82644e59ece - main - cam/iosched: Add a counter of I/Os that take too long List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: e82644e59ece5cdc67250262508e81fa22deea90 Auto-Submitted: auto-generated The branch main has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=e82644e59ece5cdc67250262508e81fa22deea90 commit e82644e59ece5cdc67250262508e81fa22deea90 Author: Warner Losh AuthorDate: 2024-07-20 02:52:40 +0000 Commit: Warner Losh CommitDate: 2024-07-20 02:53:37 +0000 cam/iosched: Add a counter of I/Os that take too long Add kern.cam.DEV.UNIT.iosched.too_long (to count I/Os taking too long) and kern.cam.DEV.UNIT.bad_latency (to set this threshold, defaults to 500ms). Each class of I/O (read, write, trim) has its own counters and thresholds. Sponsored by: Netflix Reviewed by: jhb Differential Revision: https://reviews.freebsd.org/D46033 --- sys/cam/cam_iosched.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index 471e6c355d69..022eb23cb621 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -271,6 +271,9 @@ struct iop_stats { sbintime_t emvar; sbintime_t sd; /* Last computed sd */ + uint64_t too_long; /* Number of I/Os greater than bad lat threshold */ + sbintime_t bad_latency; /* Latency threshold */ + uint32_t state_flags; #define IOP_RATE_LIMITED 1u @@ -856,6 +859,7 @@ cam_iosched_iop_stats_init(struct cam_iosched_softc *isc, struct iop_stats *ios) ios->total = 0; ios->ema = 0; ios->emvar = 0; + ios->bad_latency = SBT_1S / 2; /* Default to 500ms */ ios->softc = isc; cam_iosched_limiter_init(ios); } @@ -1046,6 +1050,15 @@ cam_iosched_iop_stats_sysctl_init(struct cam_iosched_softc *isc, struct iop_stat OID_AUTO, "errs", CTLFLAG_RD, &ios->errs, 0, "# of transactions completed with an error"); + SYSCTL_ADD_U64(ctx, n, + OID_AUTO, "too_long", CTLFLAG_RD, + &ios->too_long, 0, + "# of transactions completed took too long"); + SYSCTL_ADD_PROC(ctx, n, + OID_AUTO, "bad_latency", + CTLTYPE_STRING | CTLFLAG_RW | CTLFLAG_MPSAFE, + &ios->bad_latency, 0, cam_iosched_sbintime_sysctl, "A", + "Threshold for counting transactions that took too long (in us)"); SYSCTL_ADD_PROC(ctx, n, OID_AUTO, "limiter", @@ -1916,6 +1929,14 @@ cam_iosched_update(struct iop_stats *iop, sbintime_t sim_latency) sbintime_t y, deltasq, delta; int i; + /* + * Simple threshold: count the number of events that excede the + * configured threshold. + */ + if (sim_latency > iop->bad_latency) { + iop->too_long++; + } + /* * Keep counts for latency. We do it by power of two buckets. * This helps us spot outlier behavior obscured by averages.