From nobody Mon Feb 13 16:19:02 2023 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PFqKL4nMlz3rNkg; Mon, 13 Feb 2023 16:19:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PFqKL48G7z3MWT; Mon, 13 Feb 2023 16:19:02 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1676305142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=+nVB5L8ukv6KxfV0oo3EUQOAlUS5i21R3zgT2s24Bao=; b=sTu7Xgmzyob38zqqcWNpeCEUQCYucY5k/0XOny5bXO38li0MPSxcz/LqFATc3UH22LeqeS vhuFjf5hzkwaNkr8Ss/10TpzdVsWHSj4VDx9tN4/OH48on3B8eAPgKhPu6TMB1p5OZsMCv vNd1hzG8UIvgL4gigqW7mG/3d2uRTlQwugUJntniaWnEbe4AnTuSFVPVarFPn13znzkZjr yGquxi6JUcBOIgXu3w1JJ2v53gF+O74qPIi74NjfExu5M4fjBviADlu9pr9EQksbR5YgxB dsTVTtewxe4kd/gIF/jPYQM9oVDTR9ulM9YrAlTCXyGve9VW6IMN6HfhfSDJhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1676305142; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=+nVB5L8ukv6KxfV0oo3EUQOAlUS5i21R3zgT2s24Bao=; b=PUsZ7J1Rjk9boqf2fNgZmhLIQmwo9YvbDpVjiXRfsytRcfqRVz4w8/Pvw6qPpjNnYAICvS EdUTX9ORY6BakRF9r6MUAFYmjdEgcJrJ+TUDd6nSHzMevxuO47yCqijtLgO6aKvvB9AAyE WkOzNL8CleyG43unLntXlZ8G5dRY9hqTBB6P8RfepqBkC8v1gdbl9nYlknNOz170iiDK+V Tl9lxRxSnGcyvL4jP24H5Vp+l6r8B5XcZ6XQBPF3jO3csCM0kAe+vMV4U7/n0yF90Jkdzc SVUNtQJq5QjqSB0J4xlmEVRLwKnFqVHWsjPy40gczI80p3gl8D/+P4frmZPJ/A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1676305142; a=rsa-sha256; cv=none; b=aK1xVHkBgJvXLMcmoOLeh/xj6fktABWG9o9bibSSu2cCSqPrmQOKlOkn7d9YHRYgOv+H4Q qVr3TD3WZAv45Oqx9n8xprokvO21A2apZ1cZ6aRJvA3Vw/sgAvy4tWu1xNfcPHLiadVHE6 IcsAOzT/vv6GuqOJzLipc8EjNGtn97oIXQhZwz9ZQGkuyv7t3PEhsVURepTyfkgDT55FAs I/zFz+byqDW9OMBoPpt9/c/i+raBDoYYgyZHDnwvkG9qvwv39c772defBe1+9WETUkZs8n /ylePS+/MT0rbF+6lP+U1uuT+WgJeBzH3nGN1unBYwoG/1nqGSyRoSbJGQ9o/A== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4PFqKL3FW7zPdF; Mon, 13 Feb 2023 16:19:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 31DGJ2x0040729; Mon, 13 Feb 2023 16:19:02 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 31DGJ2EU040728; Mon, 13 Feb 2023 16:19:02 GMT (envelope-from git) Date: Mon, 13 Feb 2023 16:19:02 GMT Message-Id: <202302131619.31DGJ2EU040728@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Alan Somers Subject: git: 4c163a548080 - main - Change "ctlstat -P"'s schema List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: asomers X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 4c163a5480809d0dc8b68dd00bf2ba7d882450f9 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by asomers: URL: https://cgit.FreeBSD.org/src/commit/?id=4c163a5480809d0dc8b68dd00bf2ba7d882450f9 commit 4c163a5480809d0dc8b68dd00bf2ba7d882450f9 Author: Alan Somers AuthorDate: 2023-01-25 18:00:49 +0000 Commit: Alan Somers CommitDate: 2023-02-13 16:18:38 +0000 Change "ctlstat -P"'s schema It now reports stats separately for both ports and luns. Also, prohibit using both "-p" and "-P" at the same time. MFC after: 1 week Sponsored by: Axcient Reviewed by: mav Differential Revision: https://reviews.freebsd.org/D38500 --- usr.bin/ctlstat/ctlstat.c | 226 +++++++++++++++++++++++++++++----------------- 1 file changed, 141 insertions(+), 85 deletions(-) diff --git a/usr.bin/ctlstat/ctlstat.c b/usr.bin/ctlstat/ctlstat.c index 50b9e3b1445b..d607ab9f4523 100644 --- a/usr.bin/ctlstat/ctlstat.c +++ b/usr.bin/ctlstat/ctlstat.c @@ -50,6 +50,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -137,6 +138,7 @@ struct ctlstat_context { struct cctl_portlist_data { int level; struct sbuf *cur_sb[32]; + int id; int lun; int ntargets; char *target; @@ -149,7 +151,8 @@ struct cctl_portlist_data { static void usage(int error); static int getstats(int fd, int *alloc_items, int *num_items, - struct ctl_io_stats **xstats, struct timespec *cur_time, int *time_valid); + struct ctl_io_stats **xstats, struct timespec *cur_time, int *time_valid, + bool ports); static int getcpu(struct ctl_cpu_stats *cpu_stats); static void compute_stats(struct ctl_io_stats *cur_stats, struct ctl_io_stats *prev_stats, @@ -168,7 +171,7 @@ usage(int error) static int getstats(int fd, int *alloc_items, int *num_items, struct ctl_io_stats **stats, - struct timespec *cur_time, int *flags) + struct timespec *cur_time, int *flags, bool ports) { struct ctl_get_io_stats get_stats; int more_space_count = 0; @@ -184,8 +187,8 @@ retry: memset(*stats, 0, get_stats.alloc_len); get_stats.stats = *stats; - if (ioctl(fd, (*flags & CTLSTAT_FLAG_PORTS) ? CTL_GET_PORT_STATS : - CTL_GET_LUN_STATS, &get_stats) == -1) + if (ioctl(fd, ports ? CTL_GET_PORT_STATS : CTL_GET_LUN_STATS, + &get_stats) == -1) err(1, "CTL_GET_*_STATS ioctl returned error"); switch (get_stats.status) { @@ -395,24 +398,33 @@ ctlstat_json(struct ctlstat_context *ctx) { printf("]}"); } -#define CTLSTAT_PROMETHEUS_LOOP(field) \ +#define CTLSTAT_PROMETHEUS_LOOP(field, collector) \ for (i = n = 0; i < ctx->cur_items; i++) { \ if (F_MASK(ctx) && bit_test(ctx->item_mask, \ (int)stats[i].item) == 0) \ continue; \ for (iotype = 0; iotype < CTL_STATS_NUM_TYPES; iotype++) { \ - int lun = stats[i].item; \ - if (lun >= targdata.ntargets) \ - errx(1, "LUN %u out of range", lun); \ - printf("iscsi_target_" #field "{" \ - "lun=\"%u\",target=\"%s\",type=\"%s\"} %" PRIu64 \ + int idx = stats[i].item; \ + /* \ + * Note that Prometheus considers a label value of "" \ + * to be the same as no label at all \ + */ \ + const char *target = ""; \ + if (strcmp(collector, "port") == 0 && \ + targdata.targets[idx] != NULL) \ + { \ + target = targdata.targets[idx]; \ + } \ + printf("iscsi_%s_" #field "{" \ + "%s=\"%u\",target=\"%s\",type=\"%s\"} %" PRIu64 \ "\n", \ - lun, targdata.targets[lun], iotypes[iotype], \ + collector, collector, \ + idx, target, iotypes[iotype], \ stats[i].field[iotype]); \ } \ } \ -#define CTLSTAT_PROMETHEUS_TIMELOOP(field) \ +#define CTLSTAT_PROMETHEUS_TIMELOOP(field, collector) \ for (i = n = 0; i < ctx->cur_items; i++) { \ if (F_MASK(ctx) && bit_test(ctx->item_mask, \ (int)stats[i].item) == 0) \ @@ -420,20 +432,29 @@ ctlstat_json(struct ctlstat_context *ctx) { for (iotype = 0; iotype < CTL_STATS_NUM_TYPES; iotype++) { \ uint64_t us; \ struct timespec ts; \ - int lun = stats[i].item; \ - if (lun >= targdata.ntargets) \ - errx(1, "LUN %u out of range", lun); \ + int idx = stats[i].item; \ + /* \ + * Note that Prometheus considers a label value of "" \ + * to be the same as no label at all \ + */ \ + const char *target = ""; \ + if (strcmp(collector, "port") == 0 && \ + targdata.targets[idx] != NULL) \ + { \ + target = targdata.targets[idx]; \ + } \ bintime2timespec(&stats[i].field[iotype], &ts); \ us = ts.tv_sec * 1000000 + ts.tv_nsec / 1000; \ - printf("iscsi_target_" #field "{" \ - "lun=\"%u\",target=\"%s\",type=\"%s\"} %" PRIu64 \ + printf("iscsi_%s_" #field "{" \ + "%s=\"%u\",target=\"%s\",type=\"%s\"} %" PRIu64 \ "\n", \ - lun, targdata.targets[lun], iotypes[iotype], us); \ + collector, collector, \ + idx, target, iotypes[iotype], us); \ } \ } \ static void -cctl_start_pelement(void *user_data, const char *name, const char **attr __unused) +cctl_start_pelement(void *user_data, const char *name, const char **attr) { struct cctl_portlist_data* targdata = user_data; @@ -448,9 +469,24 @@ cctl_start_pelement(void *user_data, const char *name, const char **attr __unuse err(1, "%s: Unable to allocate sbuf", __func__); if (strcmp(name, "targ_port") == 0) { + int i = 0; + targdata->lun = -1; + targdata->id = -1; free(targdata->target); targdata->target = NULL; + while (attr[i]) { + if (strcmp(attr[i], "id") == 0) { + /* + * Well-formed XML always pairs keys with + * values in attr + */ + assert(attr[i + 1]); + targdata->id = atoi(attr[i + 1]); + } + i += 2; + } + } } @@ -486,25 +522,22 @@ cctl_end_pelement(void *user_data, const char *name) if (strcmp(name, "target") == 0) { free(targdata->target); targdata->target = str; - } else if (strcmp(name, "lun") == 0) { - targdata->lun = atoi(str); - free(str); } else if (strcmp(name, "targ_port") == 0) { - if (targdata->lun >= 0 && targdata->target != NULL) { - if (targdata->lun >= targdata->ntargets) { + if (targdata->id >= 0 && targdata->target != NULL) { + if (targdata->id >= targdata->ntargets) { /* * This can happen for example if there are - * holes in CTL's lunlist. + * targets with no LUNs. */ targdata->ntargets = MAX(targdata->ntargets * 2, - targdata->lun + 1); + targdata->id + 1); size_t newsize = targdata->ntargets * sizeof(char*); targdata->targets = rallocx(targdata->targets, newsize, MALLOCX_ZERO); } - free(targdata->targets[targdata->lun]); - targdata->targets[targdata->lun] = targdata->target; + free(targdata->targets[targdata->id]); + targdata->targets[targdata->id] = targdata->target; targdata->target = NULL; } free(str); @@ -514,7 +547,7 @@ cctl_end_pelement(void *user_data, const char *name) } static void -ctlstat_prometheus(int fd, struct ctlstat_context *ctx) { +ctlstat_prometheus(int fd, struct ctlstat_context *ctx, bool ports) { struct ctl_io_stats *stats = ctx->cur_stats; struct ctl_lun_list list; struct cctl_portlist_data targdata; @@ -522,6 +555,7 @@ ctlstat_prometheus(int fd, struct ctlstat_context *ctx) { char *port_str = NULL; int iotype, i, n, retval; int port_len = 4096; + const char *collector; bzero(&targdata, sizeof(targdata)); targdata.ntargets = ctx->cur_items; @@ -556,30 +590,23 @@ retry: } XML_ParserFree(parser); - /* - * NB: Some clients will print a warning if we don't set Content-Length, - * but they still work. And the data still gets into Prometheus. - */ - printf("HTTP/1.1 200 OK\r\n" - "Connection: close\r\n" - "Content-Type: text/plain; version=0.0.4\r\n" - "\r\n"); - - printf("# HELP iscsi_target_bytes Number of bytes\n" - "# TYPE iscsi_target_bytes counter\n"); - CTLSTAT_PROMETHEUS_LOOP(bytes); - printf("# HELP iscsi_target_dmas Number of DMA\n" - "# TYPE iscsi_target_dmas counter\n"); - CTLSTAT_PROMETHEUS_LOOP(dmas); - printf("# HELP iscsi_target_operations Number of operations\n" - "# TYPE iscsi_target_operations counter\n"); - CTLSTAT_PROMETHEUS_LOOP(operations); - printf("# HELP iscsi_target_time Cumulative operation time in us\n" - "# TYPE iscsi_target_time counter\n"); - CTLSTAT_PROMETHEUS_TIMELOOP(time); - printf("# HELP iscsi_target_dma_time Cumulative DMA time in us\n" - "# TYPE iscsi_target_dma_time counter\n"); - CTLSTAT_PROMETHEUS_TIMELOOP(dma_time); + collector = ports ? "port" : "lun"; + + printf("# HELP iscsi_%s_bytes Number of bytes\n" + "# TYPE iscsi_%s_bytes counter\n", collector, collector); + CTLSTAT_PROMETHEUS_LOOP(bytes, collector); + printf("# HELP iscsi_%s_dmas Number of DMA\n" + "# TYPE iscsi_%s_dmas counter\n", collector, collector); + CTLSTAT_PROMETHEUS_LOOP(dmas, collector); + printf("# HELP iscsi_%s_operations Number of operations\n" + "# TYPE iscsi_%s_operations counter\n", collector, collector); + CTLSTAT_PROMETHEUS_LOOP(operations, collector); + printf("# HELP iscsi_%s_time Cumulative operation time in us\n" + "# TYPE iscsi_%s_time counter\n", collector, collector); + CTLSTAT_PROMETHEUS_TIMELOOP(time, collector); + printf("# HELP iscsi_%s_dma_time Cumulative DMA time in us\n" + "# TYPE iscsi_%s_dma_time counter\n", collector, collector); + CTLSTAT_PROMETHEUS_TIMELOOP(dma_time, collector); for (i = 0; i < targdata.ntargets; i++) free(targdata.targets[i]); @@ -779,6 +806,45 @@ ctlstat_standard(struct ctlstat_context *ctx) { } } +static void +get_and_print_stats(int fd, struct ctlstat_context *ctx, bool ports) +{ + struct ctl_io_stats *tmp_stats; + int c; + + tmp_stats = ctx->prev_stats; + ctx->prev_stats = ctx->cur_stats; + ctx->cur_stats = tmp_stats; + c = ctx->prev_alloc; + ctx->prev_alloc = ctx->cur_alloc; + ctx->cur_alloc = c; + c = ctx->prev_items; + ctx->prev_items = ctx->cur_items; + ctx->cur_items = c; + ctx->prev_time = ctx->cur_time; + ctx->prev_cpu = ctx->cur_cpu; + if (getstats(fd, &ctx->cur_alloc, &ctx->cur_items, + &ctx->cur_stats, &ctx->cur_time, &ctx->flags, ports) != 0) + errx(1, "error returned from getstats()"); + + switch(ctx->mode) { + case CTLSTAT_MODE_STANDARD: + ctlstat_standard(ctx); + break; + case CTLSTAT_MODE_DUMP: + ctlstat_dump(ctx); + break; + case CTLSTAT_MODE_JSON: + ctlstat_json(ctx); + break; + case CTLSTAT_MODE_PROMETHEUS: + ctlstat_prometheus(fd, ctx, ports); + break; + default: + break; + } +} + int main(int argc, char **argv) { @@ -787,7 +853,6 @@ main(int argc, char **argv) int fd, retval; size_t size; struct ctlstat_context ctx; - struct ctl_io_stats *tmp_stats; /* default values */ retval = 0; @@ -890,10 +955,11 @@ main(int argc, char **argv) if (ctx.mode == CTLSTAT_MODE_PROMETHEUS) { if ((count != -1) || (waittime != 1) || + (F_PORTS(&ctx)) || /* NB: -P could be compatible with -t in the future */ (ctx.flags & CTLSTAT_FLAG_TOTALS)) { - errx(1, "Option -P is exclusive with -c, -w, and -t"); + errx(1, "Option -P is exclusive with -p, -c, -w, and -t"); } count = 1; } @@ -908,37 +974,27 @@ main(int argc, char **argv) if ((fd = open(CTL_DEFAULT_DEV, O_RDWR)) == -1) err(1, "cannot open %s", CTL_DEFAULT_DEV); + if (ctx.mode == CTLSTAT_MODE_PROMETHEUS) { + /* + * NB: Some clients will print a warning if we don't set + * Content-Length, but they still work. And the data still + * gets into Prometheus. + */ + printf("HTTP/1.1 200 OK\r\n" + "Connection: close\r\n" + "Content-Type: text/plain; version=0.0.4\r\n" + "\r\n"); + } + for (;count != 0;) { - tmp_stats = ctx.prev_stats; - ctx.prev_stats = ctx.cur_stats; - ctx.cur_stats = tmp_stats; - c = ctx.prev_alloc; - ctx.prev_alloc = ctx.cur_alloc; - ctx.cur_alloc = c; - c = ctx.prev_items; - ctx.prev_items = ctx.cur_items; - ctx.cur_items = c; - ctx.prev_time = ctx.cur_time; - ctx.prev_cpu = ctx.cur_cpu; - if (getstats(fd, &ctx.cur_alloc, &ctx.cur_items, - &ctx.cur_stats, &ctx.cur_time, &ctx.flags) != 0) - errx(1, "error returned from getstats()"); - - switch(ctx.mode) { - case CTLSTAT_MODE_STANDARD: - ctlstat_standard(&ctx); - break; - case CTLSTAT_MODE_DUMP: - ctlstat_dump(&ctx); - break; - case CTLSTAT_MODE_JSON: - ctlstat_json(&ctx); - break; - case CTLSTAT_MODE_PROMETHEUS: - ctlstat_prometheus(fd, &ctx); - break; - default: - break; + bool ports; + + if (ctx.mode == CTLSTAT_MODE_PROMETHEUS) { + get_and_print_stats(fd, &ctx, false); + get_and_print_stats(fd, &ctx, true); + } else { + ports = ctx.flags & CTLSTAT_FLAG_PORTS; + get_and_print_stats(fd, &ctx, ports); } fprintf(stdout, "\n");