From nobody Wed Oct 19 08:01:26 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MsjqC2HPbz4gDJv; Wed, 19 Oct 2022 08:01:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MsjqC0gTsz3xTh; Wed, 19 Oct 2022 08:01:27 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1666166487; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=eMaRIGTid8EYVgIF65C+3lY7ldvqtK6YCnKf9v3HpiI=; b=h7+Lm3xT+8T+WIVvo4hNeUE3lRRo7dO6QGq0AfGQP0Lp2C8Ug/oIC4IFIYZ13fkcuBqK+n A+xQMBIqREf/AVQ0hrbONrXDI+rFfQVsRG2pAS/7erdUGHeUktU9FLnPzwnHqYsD6XyrIs Q6FPLuueXoEWSgLwUMwoxQBp56KCKBr5ARaOSla5gEnO1mOKqy5+vzQqEj/eMX89kcxekK v1LxkU/WiID66k9u+7A9ZG8JZKOb3KNBtLub6xK/a7DkSbi3+eoFE9PKYs+8Y95/n15J+D wLoc8R5r7SXPqcIyD8Yg2iLxEINh8UWHJHMXu4ZqBTRWTnx2QL84fuODlAGa7A== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4MsjqB5tF2zSQq; Wed, 19 Oct 2022 08:01:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 29J81Qu4060637; Wed, 19 Oct 2022 08:01:26 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 29J81QJb060636; Wed, 19 Oct 2022 08:01:26 GMT (envelope-from git) Date: Wed, 19 Oct 2022 08:01:26 GMT Message-Id: <202210190801.29J81QJb060636@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Baptiste Daroussin Subject: git: 489c9df19851 - stable/13 - sort: replace home made line reader by getdelim(3) List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: bapt X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 489c9df19851aa052553a79de65e37ea1471014e Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1666166487; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=eMaRIGTid8EYVgIF65C+3lY7ldvqtK6YCnKf9v3HpiI=; b=tba4wEnizodE4UKCWb+Q87anXXoKlO97I6V96tM0xjoIpmdsxs5NXxhPujUQtU+kKGFMQD hM/KPm5DZgnBD8HW+wKvLkT3i6sBU6We3h7BYX8gjEC6Y92UHUzJP2jnDd+MQ3n5qn4i4e +GDt8tetpUD51+Ab84pY6geMT2TLX/2ce79l6LO+Xj0Vv2TABYDYzsi45xdohGFmcm3gqA fSmVLfZNTTtuW1qCWX3ysTRnBagZZHPsBIsN4UKCDaYuvzblqksrQAbxeJFLAnnRBa03BY Mhm3qwM4K1OCjZb2c9ZHL/hs2WHOCy78MDac5Ah1JtoaJJ2LjKyPtvqGV1mQPA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1666166487; a=rsa-sha256; cv=none; b=yZH3WxUm8lZiGe4Wio1raGu3BPSCcARu0luuA5TFWzY+0zqmjB7Uno1AM4zMh8QcCh0ZUI pjzLsN2alpGQEavDM+a5hRpgvKmWJNdVJ4I1r3ygjtz8B2Bx+xuBGS7hu9OHiHXkujY1mx +6gByCpHAWi0IFJxmNJT6JMd/2SxgiKk4D/HDmidWzw3I1IJ0oHyUR/DJe8OhZeTQ1EfPc +AOkGT7FXMfW3yfqfeadkmXqfy9ynyY68JNjSIx/M0575D+zG1D5/eQXBB436pkw5g01dA IwoUKHxxenk2VZa5gH9VqbRmGhE16qNVN17I/C39ELuAmqBtNcix69B1ISxwIg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by bapt: URL: https://cgit.FreeBSD.org/src/commit/?id=489c9df19851aa052553a79de65e37ea1471014e commit 489c9df19851aa052553a79de65e37ea1471014e Author: Baptiste Daroussin AuthorDate: 2022-10-12 13:31:50 +0000 Commit: Baptiste Daroussin CommitDate: 2022-10-19 07:59:28 +0000 sort: replace home made line reader by getdelim(3) The previous code had bug when reading lines with an unexpected encoding, returning without the full line being captured. This result in sort complaining with "sort: Illegal byte sequence" Using getdelim(3) instead of the home made code, fixes the situation. PR: 241679 Reported by: Ronald F. Guilmette MFC After: 1 week Reviewed by: markj, imp Differential Revision: https://reviews.freebsd.org/D36948 (cherry picked from commit b58094c0d98e5d0ab8abdcdb655ac902ae8ad66e) --- usr.bin/sort/bwstring.c | 109 ------------------------------------------------ usr.bin/sort/file.c | 18 +++++--- 2 files changed, 12 insertions(+), 115 deletions(-) diff --git a/usr.bin/sort/bwstring.c b/usr.bin/sort/bwstring.c index 2f2737e94314..52173002986f 100644 --- a/usr.bin/sort/bwstring.c +++ b/usr.bin/sort/bwstring.c @@ -470,115 +470,6 @@ bwsfwrite(struct bwstring *bws, FILE *f, bool zero_ended) } } -/* - * Allocate and read a binary string from file. - * The strings are nl-ended or zero-ended, depending on the sort setting. - */ -struct bwstring * -bwsfgetln(FILE *f, size_t *len, bool zero_ended, struct reader_buffer *rb) -{ - wint_t eols; - - eols = zero_ended ? btowc('\0') : btowc('\n'); - - if (!zero_ended && (mb_cur_max > 1)) { - wchar_t *ret; - - ret = fgetwln(f, len); - - if (ret == NULL) { - if (!feof(f)) - err(2, NULL); - return (NULL); - } - if (*len > 0) { - if (ret[*len - 1] == (wchar_t)eols) - --(*len); - } - return (bwssbdup(ret, *len)); - - } else if (!zero_ended && (mb_cur_max == 1)) { - char *ret; - - ret = fgetln(f, len); - - if (ret == NULL) { - if (!feof(f)) - err(2, NULL); - return (NULL); - } - if (*len > 0) { - if (ret[*len - 1] == '\n') - --(*len); - } - return (bwscsbdup((unsigned char *)ret, *len)); - - } else { - *len = 0; - - if (feof(f)) - return (NULL); - - if (2 >= rb->fgetwln_z_buffer_size) { - rb->fgetwln_z_buffer_size += 256; - rb->fgetwln_z_buffer = sort_realloc(rb->fgetwln_z_buffer, - sizeof(wchar_t) * rb->fgetwln_z_buffer_size); - } - rb->fgetwln_z_buffer[*len] = 0; - - if (mb_cur_max == 1) - while (!feof(f)) { - int c; - - c = fgetc(f); - - if (c == EOF) { - if (*len == 0) - return (NULL); - goto line_read_done; - } - if (c == eols) - goto line_read_done; - - if (*len + 1 >= rb->fgetwln_z_buffer_size) { - rb->fgetwln_z_buffer_size += 256; - rb->fgetwln_z_buffer = sort_realloc(rb->fgetwln_z_buffer, - SIZEOF_WCHAR_STRING(rb->fgetwln_z_buffer_size)); - } - - rb->fgetwln_z_buffer[*len] = c; - rb->fgetwln_z_buffer[++(*len)] = 0; - } - else - while (!feof(f)) { - wint_t c; - - c = fgetwc(f); - - if (c == WEOF) { - if (*len == 0) - return (NULL); - goto line_read_done; - } - if (c == eols) - goto line_read_done; - - if (*len + 1 >= rb->fgetwln_z_buffer_size) { - rb->fgetwln_z_buffer_size += 256; - rb->fgetwln_z_buffer = sort_realloc(rb->fgetwln_z_buffer, - SIZEOF_WCHAR_STRING(rb->fgetwln_z_buffer_size)); - } - - rb->fgetwln_z_buffer[*len] = c; - rb->fgetwln_z_buffer[++(*len)] = 0; - } - -line_read_done: - /* we do not count the last 0 */ - return (bwssbdup(rb->fgetwln_z_buffer, *len)); - } -} - int bwsncmp(const struct bwstring *bws1, const struct bwstring *bws2, size_t offset, size_t len) diff --git a/usr.bin/sort/file.c b/usr.bin/sort/file.c index 1fb02642b5c9..ffcd71e9f991 100644 --- a/usr.bin/sort/file.c +++ b/usr.bin/sort/file.c @@ -75,7 +75,7 @@ struct file_reader struct reader_buffer rb; FILE *file; char *fname; - unsigned char *buffer; + char *buffer; unsigned char *mmapaddr; unsigned char *mmapptr; size_t bsz; @@ -713,7 +713,7 @@ file_reader_readline(struct file_reader *fr) } } else if (fr->file != stdin) { - unsigned char *strend; + char *strend; size_t bsz1, remsz, search_start; search_start = 0; @@ -785,10 +785,16 @@ file_reader_readline(struct file_reader *fr) fr->strbeg = (strend - fr->buffer) + 1; } else { - size_t len = 0; - - ret = bwsfgetln(fr->file, &len, sort_opts_vals.zflag, - &(fr->rb)); + int delim = sort_opts_vals.zflag ? '\0' : '\n'; + ssize_t len = getdelim(&fr->buffer, &fr->bsz, delim, fr->file); + if (len < 0) { + if (!feof(fr->file)) + err(2, NULL); + return (NULL); + } + if (len > 0 && fr->buffer[len - 1] == delim) + len--; + ret = bwscsbdup(fr->buffer, len); } return (ret);