From nobody Wed Aug 23 03:41:17 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RVsSs2yNfz4qtG2; Wed, 23 Aug 2023 03:41:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RVsSs2FCwz3SS1; Wed, 23 Aug 2023 03:41:17 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692762077; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Gb6SOKTJ5kBPZKq/XMFiTz7iBeyAu3t+NBwFBgDZuNI=; b=GklkGuZhVlfXey533hhUMVZo7TCnD6Myma8/PGlAngQ9qfFHNJv8VL4ZKI9cBxDILg2h+3 FlBRMaCNn3FWuLvl7fyUILXVC65BBPwIWLbJPEhZo/fiP7mkntmz47Rh9b/O2USVp6lifv mF3D1isnRMvzvNckLEkaGqCP6r/R9IMpwjANkQLpNdR9cnKcXc3K8XzA22YJidhMTHzXRz XchZ3FUvoM1QlnM5nDQ9mxX9xuwsAr3ZT4/+79O8Elr11Rn3CDjHxQeoSXlHCNP8k0r6Rx FSDylDTWN0MdB2W38j72YxCqu1u9l3QTX7DcOZD8hewQvMFIPl44F13YB8TrWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692762077; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Gb6SOKTJ5kBPZKq/XMFiTz7iBeyAu3t+NBwFBgDZuNI=; b=ZYMtgjchqQDgygpI5wNyJVnQtTiV2+toJ4/T9Fh2bs0469G5H6OcP+7sjelaw/p9kkNMws Dk7l6sZc0i7sMW8pIFox5GKMRf7tBhzQEY96xzyBi4GHIUbsYdFrJ114g7aDay6OIskGXe TyXhwk96q/1ciU1GwiLYsumdPF8Z5a9MKrqoAF3D2VIofH6z0wlay7D6Nc5DJ7ZLewHQlP RguVNUyCMvg/p97bwfWlF2hHFLdd6NlLLAjSVDXVYtxmkaE6iODlE+DE0/imSvQ3JG9dsl Dyp1VX7SdsgBgkvHc81PDACSAQC7iBit1jGZu/PSJG+2nKx0bZhGeX9hjwBLfw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692762077; a=rsa-sha256; cv=none; b=oa1c7/9ehNl4QZ/YBch8J24b4H7e0YRnkPeZULG9IuHeW+8BGukGirm/eLzvta39pvu1i5 MNPmg7zf+FwznWf0iTRcdJLr1Lazl73Dsa3qZ9S+VV3tajMghA7bTmvhLko7+DIdmxoJPO aDot02o3xNp19blpikiZ4/96naIAR5E+nyYBzouGYiCIBZCjLcW2SVpiV/6XYZqxDo7qZR zvW0PqEa1vnM67dRbDou2PQx/cuALa0vu0IVaxP+iTlmNSmuQem7UL4YwJnzilOL3C7T3f g8ZlpLu+OMxRnQoks8ioMhuHmKEiWq3ZS8QLzptJ3Y9LkJgF5/tJKRC9YYi/qg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4RVsSs10RtzVXZ; Wed, 23 Aug 2023 03:41:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 37N3fHYG041096; Wed, 23 Aug 2023 03:41:17 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 37N3fHOO041093; Wed, 23 Aug 2023 03:41:17 GMT (envelope-from git) Date: Wed, 23 Aug 2023 03:41:17 GMT Message-Id: <202308230341.37N3fHOO041093@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Kyle Evans Subject: git: e2030ca246a7 - main - libc: fix c*rtomb/mbrtoc* List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kevans X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: e2030ca246a777c50a7aacb3a35f88addded21cf Auto-Submitted: auto-generated The branch main has been updated by kevans: URL: https://cgit.FreeBSD.org/src/commit/?id=e2030ca246a777c50a7aacb3a35f88addded21cf commit e2030ca246a777c50a7aacb3a35f88addded21cf Author: Kyle Evans AuthorDate: 2023-08-23 03:40:45 +0000 Commit: Kyle Evans CommitDate: 2023-08-23 03:40:45 +0000 libc: fix c*rtomb/mbrtoc* In 693f88c9da8d ("iconv_std: complete the //IGNORE support"), we more completely implemented //IGNORE, which changed the semantics of ci_discard_ilseq. DISCARD_ILSEQ semantics are supposed to match //IGNORE, so we really can't do much about that particular incompatibility. This broke c*rtomb and mbrtoc* handling of invalid sequences, but it turns out they don't want DISCARD_ILSEQ semantics at all; they really want the subset that we call _CITRUS_ICONV_F_HIDE_INVALID. This restores the exact flow in iconv_std to precisely how it happened prior to 693f88c9da8d. PR: 265871 Fixes: 693f88c9da8d ("iconv_std: complete the //IGNORE support") Sponsored by: Klara, Inc. Differential Revision: https://reviews.freebsd.org/D41513 --- lib/libc/locale/cXXrtomb_iconv.h | 4 ++-- lib/libc/locale/mbrtocXX_iconv.h | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/lib/libc/locale/cXXrtomb_iconv.h b/lib/libc/locale/cXXrtomb_iconv.h index cf171a49d439..8f123e85c66d 100644 --- a/lib/libc/locale/cXXrtomb_iconv.h +++ b/lib/libc/locale/cXXrtomb_iconv.h @@ -75,7 +75,7 @@ cXXrtomb_l(char * __restrict s, charXX_t c, mbstate_t * __restrict ps, errno = EINVAL; return (-1); } - handle->cv_shared->ci_discard_ilseq = true; + handle->cv_shared->ci_discard_ilseq = false; handle->cv_shared->ci_hooks = NULL; cs->srcbuf_len = 0; cs->initialized = true; @@ -92,7 +92,7 @@ cXXrtomb_l(char * __restrict s, charXX_t c, mbstate_t * __restrict ps, dst = s; dstleft = MB_CUR_MAX_L(locale); err = _citrus_iconv_convert(handle, &src, &srcleft, &dst, &dstleft, - 0, &invlen); + _CITRUS_ICONV_F_HIDE_INVALID, &invlen); /* Character is part of a surrogate pair. We need more input. */ if (err == EINVAL) diff --git a/lib/libc/locale/mbrtocXX_iconv.h b/lib/libc/locale/mbrtocXX_iconv.h index 148ba55e548a..22b8a7d4031c 100644 --- a/lib/libc/locale/mbrtocXX_iconv.h +++ b/lib/libc/locale/mbrtocXX_iconv.h @@ -78,7 +78,7 @@ mbrtocXX_l(charXX_t * __restrict pc, const char * __restrict s, size_t n, errno = EINVAL; return (-1); } - handle->cv_shared->ci_discard_ilseq = true; + handle->cv_shared->ci_discard_ilseq = false; handle->cv_shared->ci_hooks = NULL; cs->srcbuf_len = cs->dstbuf_len = 0; cs->initialized = true; @@ -110,7 +110,7 @@ mbrtocXX_l(charXX_t * __restrict pc, const char * __restrict s, size_t n, assert(srcleft <= sizeof(cs->srcbuf) && dstleft <= sizeof(cs->dstbuf.bytes)); err = _citrus_iconv_convert(handle, &src, &srcleft, - &dst, &dstleft, 0, &invlen); + &dst, &dstleft, _CITRUS_ICONV_F_HIDE_INVALID, &invlen); cs->dstbuf_len = (dst - cs->dstbuf.bytes) / sizeof(charXX_t); /* Got new character(s). Return the first. */