From nobody Sat Jul 20 02:59:09 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WQrq33fBCz5QxxJ; Sat, 20 Jul 2024 02:59:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WQrq16N41z438T; Sat, 20 Jul 2024 02:59:09 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721444349; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=FuoJEx59qZqxdCCHexYQhMDYWsY+BuIZj3lGFGIClq4=; b=nOB9ZUWGDqn/JYNWpM5vdE09z2HEgNVYzBzv5c4qVTwHkGg7sRkSb0qwX3LCFWe62g6e8p xXlUwAk0X8f3JG/jLNaWmvnuQnasRKM3qrHXGKMXxXN844BuDwJQiOoYqDEliwtM6qETz+ B+PnDlNah/AFJxJonyoHSsfRoQ2Ukx5H8T3gmsUitL+fTh5rr2lCr+YfWPAYxwk/p7KwxY PGRRbQjh1vus1LIrIQCVYy9qNLmUVugPJFr/fyiB6SmnXlnIElnFa+aDG5OknCC3OBIiDr Na348PO9X4tgVJDbHVIFq7dNjzRhkO0w8ORAS91+8VPQPu9K3UB33bD7i6k8Uw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1721444349; a=rsa-sha256; cv=none; b=QkuXJEv7vEBE2v+B7/djyMuo8Le4QbkQepwO1A8nlYqCzW/yoHUAnFKOrFw66L+VWj+cDH i13cKzvEBA9ijjBId9ya/d37zcthxHIbGytuxgh1sF2Nxm3/aO5N5XMlt6IGKRjTvzZNby V32+6ob9EgXb1U41svZdkS/8ur0xzmgRccdJml+xfQoahCTpdla+4jW44i/Pjg45kyAyhk wIvu/gFVVGFHDtSs7fKG8SpH6e40zzMEKFEf+IffRJPQbUuu7cm/xVFgjU3skPMt8DDdue HwqhXMbVgjyN5kzewgvaUwaNJ+yIhcKhgDgymF0/k/gAEZ7VuJz7Xc/H/BxcIw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1721444349; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=FuoJEx59qZqxdCCHexYQhMDYWsY+BuIZj3lGFGIClq4=; b=VX451jb2m+6oZz2MGsG22vKQVKtkLCJuvIU3fsJYaQTlnZoYPxrnts1lqn70RXasFkNea5 q81PN3kfCB6JZiTppJDsp/06hVCuFCAIZ+olPuLcBdY+99D6Sc3srheIfoO2sRNkZneKGb mf+B1Pjmq/GKKjWby1jCNi9WgqzBS1Xp4xl0z40dpKBvVsKt/e5i3Q/ly0Ifdpeq0OVkWS 3BOXPbOhwD0rRHkhUGfPinyW5TCcVBsmeWMmvk35YGxjGRcqVgzuIEzJHYtkMwYhLOs1Da twGFcTsT8rdcRlirIlDdqA0e/cbTh2aXTdY6SEf2xvyWffUxST/HGFs3ki/aTQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WQrq15mtvzykQ; Sat, 20 Jul 2024 02:59:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 46K2x9At024549; Sat, 20 Jul 2024 02:59:09 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 46K2x9Uv024546; Sat, 20 Jul 2024 02:59:09 GMT (envelope-from git) Date: Sat, 20 Jul 2024 02:59:09 GMT Message-Id: <202407200259.46K2x9Uv024546@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Warner Losh Subject: git: dd6123ebf02b - main - cam/iosched: Log outlier latency events List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: dd6123ebf02b86cee90b75e54fc755f6c20caebf Auto-Submitted: auto-generated The branch main has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=dd6123ebf02b86cee90b75e54fc755f6c20caebf commit dd6123ebf02b86cee90b75e54fc755f6c20caebf Author: Warner Losh AuthorDate: 2024-07-20 02:53:08 +0000 Commit: Warner Losh CommitDate: 2024-07-20 02:53:37 +0000 cam/iosched: Log outlier latency events Log outlier latency events to devd. In addition to counting, this will allow analysis of whether the problem is confined to a specific block range, or if it's a more general problem. Sponsored by: Netflix Reviewed by: jhb Differential Revision: https://reviews.freebsd.org/D46036 --- sys/cam/cam_iosched.c | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/sys/cam/cam_iosched.c b/sys/cam/cam_iosched.c index cfefc8098d18..28b5375ae246 100644 --- a/sys/cam/cam_iosched.c +++ b/sys/cam/cam_iosched.c @@ -1813,7 +1813,6 @@ cam_iosched_bio_complete(struct cam_iosched_softc *isc, struct bio *bp, if (isc->latfcn && isc->max_lat != 0 && sim_latency > isc->max_lat) isc->latfcn(isc->latarg, sim_latency, bp); } - #endif return retval; } @@ -1926,9 +1925,35 @@ static sbintime_t latencies[LAT_BUCKETS - 1] = { BUCKET_BASE << 18 /* 5,242,880us */ }; +#define CAM_IOSCHED_DEVD_MSG_SIZE 256 + +static void +cam_iosched_devctl_outlier(struct iop_stats *iop, sbintime_t sim_latency, + const struct bio *bp) +{ + daddr_t lba = bp->bio_pblkno; + daddr_t cnt = bp->bio_bcount / iop->softc->disk->d_sectorsize; + char *sbmsg; + struct sbuf sb; + + sbmsg = malloc(CAM_IOSCHED_DEVD_MSG_SIZE, M_CAMSCHED, M_NOWAIT); + if (sbmsg == NULL) + return; + sbuf_new(&sb, sbmsg, CAM_IOSCHED_DEVD_MSG_SIZE, SBUF_FIXEDLEN); + + sbuf_printf(&sb, "device=%s%d lba=%jd blocks=%jd latency=%jd", + iop->softc->periph->periph_name, + iop->softc->periph->unit_number, + lba, cnt, sbttons(sim_latency)); + if (sbuf_finish(&sb) == 0) + devctl_notify("CAM", "iosched", "latency", sbuf_data(&sb)); + sbuf_delete(&sb); + free(sbmsg, M_CAMSCHED); +} + static void cam_iosched_update(struct iop_stats *iop, sbintime_t sim_latency, - const struct bio *bp __unused) + const struct bio *bp) { sbintime_t y, deltasq, delta; int i; @@ -1938,6 +1963,7 @@ cam_iosched_update(struct iop_stats *iop, sbintime_t sim_latency, * configured threshold. */ if (sim_latency > iop->bad_latency) { + cam_iosched_devctl_outlier(iop, sim_latency, bp); iop->too_long++; }