From nobody Sun Aug 13 08:43:32 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RNrfD2YgWz4qDlb; Sun, 13 Aug 2023 08:43:32 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RNrfD1kymz4T9r; Sun, 13 Aug 2023 08:43:32 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691916212; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1AvTTIuyx6hlFHe8kw0wseGmtPqT/sbi2hvb8w6anEY=; b=eH9TDdu6VPmsRjJ2vMWhO/cUNoUNnr86lCduJvY5uc7TOEYmEoPhXYe5JC2Ri4J/H0h3Nl ApwFGWHqxMubzoF2rzaj41Oc+sTsguxQ038qYVT7LMrIc+ijP+aNO1QdCw/mj2VUJKG0cK 6ZIqbJDACFpmn4jt93IY2og2qjOgfAYpfFjs4oMR/GKCV2W+RGZMbcDGDsG9tabJp2lABA 6RISjRt1ZXXOoxEllRKaOqG8HswrmcbyjtzkY+wQ+KoJXeSGGyPp/3Wt+hBvFQ4yiebKkz G3t6+aXUccrv0wTaO4X4f65sLSl08xxuD13wqrsQEO/i/JYYROjI4+e7CrHmYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691916212; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1AvTTIuyx6hlFHe8kw0wseGmtPqT/sbi2hvb8w6anEY=; b=whXKNSzVYyktdxpCdm7bdJqTOvkc+jtenBxwTrMOVyOEFxjIniXw8jYrEb6uHP2mjSvaO7 Kgoq7h+c3P3BXmeDKdZSq76V0y/A/xRK7z9sFxdONZF6XyAbaIaft+lwchnnNxRDeFAUX9 H//DrUzpmFGC59BgvMQlWrTUxhdRWvYlWS9eJOKjMUzzTHAeSLM0iN3tNCCHHNj3fsKTmT 8H6TITfwBGBBBR4qpEtEzyG/8MMFMkYPLej3OTkriWSi+j4Rhe3SqwlxGmdjftpJ/P9WvH Jp3KKBOLd2F1JWogy4zud5ngjgqPRTP1eiOpeelulNIF9qDmGMS+L3FmIQPJgA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1691916212; a=rsa-sha256; cv=none; b=iySbvloMfcIZvJlTjbrvy2WMPNu7nuWCqosjrIvvCk7N1PCtuT33FQVGSHQ6K0ypIaphUD ft1OWkvN85bne9vHtB521Nqjic+AFhpplrsgmWb7zsB8cjoQSJ+C+AWObExZspEhPYDuh8 8XM+nvKgZDUcSiNCMCVC/MUzJiV4j6DPO0WfF5mYgplw7hydfUfoPnUULVUflkhtP0PsAk lySAQ79wsr64W8vlM7M5NNtO7RCkWa636ug+47xehoeXZ2n3Y7zsjJcycTmcTMx/9JHw8b PkMi35N3pBmfvXbn+uJYU1o+3+SF0ITVo3mcew5EioY4Q5OS6jKObu0yZ1Xn0w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4RNrfD0pHBzwHQ; Sun, 13 Aug 2023 08:43:32 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 37D8hWbO051548; Sun, 13 Aug 2023 08:43:32 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 37D8hWER051545; Sun, 13 Aug 2023 08:43:32 GMT (envelope-from git) Date: Sun, 13 Aug 2023 08:43:32 GMT Message-Id: <202308130843.37D8hWER051545@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Dimitry Andric Subject: git: dd565d99901a - stable/12 - Use a builtin where possible in msun List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: dim X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: dd565d99901a891192e0b821cab5f9523c603f38 Auto-Submitted: auto-generated The branch stable/12 has been updated by dim: URL: https://cgit.FreeBSD.org/src/commit/?id=dd565d99901a891192e0b821cab5f9523c603f38 commit dd565d99901a891192e0b821cab5f9523c603f38 Author: Andrew Turner AuthorDate: 2021-11-02 11:31:17 +0000 Commit: Dimitry Andric CommitDate: 2023-08-13 08:39:08 +0000 Use a builtin where possible in msun Some of the functions in msun can be implemented using a compiler builtin function to generate a small number of instructions. Implement this support in fma, fmax, fmin, and sqrt on arm64. Care must be taken as the builtin can be implemented as a function call on some architectures that lack direct support. In these cases we need to use the original code path. As we don't set errno on failure build with -fno-math-errno so the toolchain doesn't convert a builtin into a function call when it detects a failure, e.g. gcc will add a call to sqrt when the input is negative leading to an infinite loop. Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D32801 (cherry picked from commit b2e843161dc3b79777e873183447c92ed9c3703a) --- lib/msun/Makefile | 5 +++++ lib/msun/aarch64/Makefile.inc | 12 ++++++++++++ lib/msun/src/e_sqrt.c | 18 +++++++++++++----- lib/msun/src/e_sqrtf.c | 8 ++++++++ lib/msun/src/s_fma.c | 8 ++++++++ lib/msun/src/s_fmaf.c | 8 ++++++++ lib/msun/src/s_fmax.c | 8 ++++++++ lib/msun/src/s_fmaxf.c | 8 ++++++++ lib/msun/src/s_fmin.c | 8 ++++++++ lib/msun/src/s_fminf.c | 8 ++++++++ 10 files changed, 86 insertions(+), 5 deletions(-) diff --git a/lib/msun/Makefile b/lib/msun/Makefile index ae8b42bebe40..4771de91c561 100644 --- a/lib/msun/Makefile +++ b/lib/msun/Makefile @@ -45,6 +45,11 @@ CFLAGS+= -I${.CURDIR}/${ARCH_SUBDIR} CFLAGS+= -ffp-exception-behavior=maytrap .endif +# Tell the compiler we don't set errno in any of the math functions. This +# stops gcc from trying to generate a function call to set errno when using +# a builtin in the implementation +CFLAGS+= -fno-math-errno + .PATH: ${.CURDIR}/bsdsrc .PATH: ${.CURDIR}/src .PATH: ${.CURDIR}/man diff --git a/lib/msun/aarch64/Makefile.inc b/lib/msun/aarch64/Makefile.inc index 286a60859591..b0e01488b0bb 100644 --- a/lib/msun/aarch64/Makefile.inc +++ b/lib/msun/aarch64/Makefile.inc @@ -2,3 +2,15 @@ LDBL_PREC = 113 +# Use a builtin when it generates the needed instruction +CFLAGS+=-DUSE_BUILTIN_FMAF +CFLAGS+=-DUSE_BUILTIN_FMA + +CFLAGS+=-DUSE_BUILTIN_FMAXF +CFLAGS+=-DUSE_BUILTIN_FMAX + +CFLAGS+=-DUSE_BUILTIN_FMINF +CFLAGS+=-DUSE_BUILTIN_FMIN + +CFLAGS+=-DUSE_BUILTIN_SQRTF +CFLAGS+=-DUSE_BUILTIN_SQRT diff --git a/lib/msun/src/e_sqrt.c b/lib/msun/src/e_sqrt.c index 12fb56e1340b..37351a425ce4 100644 --- a/lib/msun/src/e_sqrt.c +++ b/lib/msun/src/e_sqrt.c @@ -14,6 +14,18 @@ #include __FBSDID("$FreeBSD$"); +#include + +#include "math.h" +#include "math_private.h" + +#ifdef USE_BUILTIN_SQRT +double +__ieee754_sqrt(double x) +{ + return (__builtin_sqrt(x)); +} +#else /* __ieee754_sqrt(x) * Return correctly rounded sqrt. * ------------------------------------------ @@ -84,11 +96,6 @@ __FBSDID("$FreeBSD$"); *--------------- */ -#include - -#include "math.h" -#include "math_private.h" - static const double one = 1.0, tiny=1.0e-300; double @@ -187,6 +194,7 @@ __ieee754_sqrt(double x) INSERT_WORDS(z,ix0,ix1); return z; } +#endif #if (LDBL_MANT_DIG == 53) __weak_reference(sqrt, sqrtl); diff --git a/lib/msun/src/e_sqrtf.c b/lib/msun/src/e_sqrtf.c index 7eba4d07f92c..06e5d62fa5f9 100644 --- a/lib/msun/src/e_sqrtf.c +++ b/lib/msun/src/e_sqrtf.c @@ -20,6 +20,13 @@ static char rcsid[] = "$FreeBSD$"; #include "math.h" #include "math_private.h" +#ifdef USE_BUILTIN_SQRTF +float +__ieee754_sqrtf(float x) +{ + return (__builtin_sqrtf(x)); +} +#else static const float one = 1.0, tiny=1.0e-30; float @@ -87,3 +94,4 @@ __ieee754_sqrtf(float x) SET_FLOAT_WORD(z,ix); return z; } +#endif diff --git a/lib/msun/src/s_fma.c b/lib/msun/src/s_fma.c index 41a6424fdf7f..95cffd0ba36d 100644 --- a/lib/msun/src/s_fma.c +++ b/lib/msun/src/s_fma.c @@ -35,6 +35,13 @@ __FBSDID("$FreeBSD$"); #include "math_private.h" +#ifdef USE_BUILTIN_FMA +double +fma(double x, double y, double z) +{ + return (__builtin_fma(x, y, z)); +} +#else /* * A struct dd represents a floating-point number with twice the precision * of a double. We maintain the invariant that "hi" stores the 53 high-order @@ -284,6 +291,7 @@ fma(double x, double y, double z) else return (add_and_denormalize(r.hi, adj, spread)); } +#endif /* !USE_BUILTIN_FMA */ #if (LDBL_MANT_DIG == 53) __weak_reference(fma, fmal); diff --git a/lib/msun/src/s_fmaf.c b/lib/msun/src/s_fmaf.c index 389cf1b11bee..4591cc219bbb 100644 --- a/lib/msun/src/s_fmaf.c +++ b/lib/msun/src/s_fmaf.c @@ -34,6 +34,13 @@ __FBSDID("$FreeBSD$"); #include "math.h" #include "math_private.h" +#ifdef USE_BUILTIN_FMAF +float +fmaf(float x, float y, float z) +{ + return (__builtin_fmaf(x, y, z)); +} +#else /* * Fused multiply-add: Compute x * y + z with a single rounding error. * @@ -69,3 +76,4 @@ fmaf(float x, float y, float z) SET_LOW_WORD(adjusted_result, lr + 1); return (adjusted_result); } +#endif /* !USE_BUILTIN_FMAF */ diff --git a/lib/msun/src/s_fmax.c b/lib/msun/src/s_fmax.c index 0c234bcc5f7f..b53b1e64da04 100644 --- a/lib/msun/src/s_fmax.c +++ b/lib/msun/src/s_fmax.c @@ -34,6 +34,13 @@ __FBSDID("$FreeBSD$"); #include "fpmath.h" +#ifdef USE_BUILTIN_FMAX +double +fmax(double x, double y) +{ + return (__builtin_fmax(x, y)); +} +#else double fmax(double x, double y) { @@ -54,6 +61,7 @@ fmax(double x, double y) return (x > y ? x : y); } +#endif #if (LDBL_MANT_DIG == 53) __weak_reference(fmax, fmaxl); diff --git a/lib/msun/src/s_fmaxf.c b/lib/msun/src/s_fmaxf.c index 8e9d1baf3e8e..8d3d14f4b52a 100644 --- a/lib/msun/src/s_fmaxf.c +++ b/lib/msun/src/s_fmaxf.c @@ -33,6 +33,13 @@ __FBSDID("$FreeBSD$"); #include "fpmath.h" +#ifdef USE_BUILTIN_FMAXF +float +fmaxf(float x, float y) +{ + return (__builtin_fmaxf(x, y)); +} +#else float fmaxf(float x, float y) { @@ -53,3 +60,4 @@ fmaxf(float x, float y) return (x > y ? x : y); } +#endif diff --git a/lib/msun/src/s_fmin.c b/lib/msun/src/s_fmin.c index d7f24c121fb2..53f36c1c2d3d 100644 --- a/lib/msun/src/s_fmin.c +++ b/lib/msun/src/s_fmin.c @@ -34,6 +34,13 @@ __FBSDID("$FreeBSD$"); #include "fpmath.h" +#ifdef USE_BUILTIN_FMIN +double +fmin(double x, double y) +{ + return (__builtin_fmin(x, y)); +} +#else double fmin(double x, double y) { @@ -54,6 +61,7 @@ fmin(double x, double y) return (x < y ? x : y); } +#endif #if (LDBL_MANT_DIG == 53) __weak_reference(fmin, fminl); diff --git a/lib/msun/src/s_fminf.c b/lib/msun/src/s_fminf.c index 258316725ddd..58b6a489b820 100644 --- a/lib/msun/src/s_fminf.c +++ b/lib/msun/src/s_fminf.c @@ -33,6 +33,13 @@ __FBSDID("$FreeBSD$"); #include "fpmath.h" +#ifdef USE_BUILTIN_FMINF +float +fminf(float x, float y) +{ + return (__builtin_fminf(x, y)); +} +#else float fminf(float x, float y) { @@ -53,3 +60,4 @@ fminf(float x, float y) return (x < y ? x : y); } +#endif