From nobody Sat Aug 31 15:56:07 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wx04P3f6Fz5MbPX for ; Sat, 31 Aug 2024 15:56:21 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ot1-x336.google.com (mail-ot1-x336.google.com [IPv6:2607:f8b0:4864:20::336]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wx04N3WgHz4HvJ for ; Sat, 31 Aug 2024 15:56:20 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20230601.gappssmtp.com header.s=20230601 header.b=BDHFiJf4; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::336) smtp.mailfrom=wlosh@bsdimp.com Received: by mail-ot1-x336.google.com with SMTP id 46e09a7af769-70df4b5cdd8so1894716a34.1 for ; Sat, 31 Aug 2024 08:56:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1725119779; x=1725724579; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=75fxV43JOTZiFhd8EWNGWULZ7A1AQ7/H3XkK710mKb8=; b=BDHFiJf4cG8oSqeqCoKLEax5gg1/KW21SjRhRF6ftIrMydhV3nKqAmVI+6lODMqFLk R/XkcJsGeQS/TdNX0fe7lZzxD5w8dY+/2oo+l+pvK9ozScffWHGXzC1seJ8SElV9abZc +6R65cdS9C4Nv9H39Cex/Kst14So+qHobgolkLpDpVqDag9ymJ2nNhoeAfHTEO/BO/DL xvmeNIdz9a3eUJwTVqlBu8/1iWz5xkFXzJDBjw8v+4Ht3so11gFIDrTD5OcHoUm3zdA3 iwLvBT2NTQfFaDTAX23cTVk7C8/w4tBYMjyHZM/XKh8EERwjUHYEPcyZmEkCbc0M40Rb Iujw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725119779; x=1725724579; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=75fxV43JOTZiFhd8EWNGWULZ7A1AQ7/H3XkK710mKb8=; b=urNsySuKA8sFqzvp0mid5gGr3+TOnpNPVlznYAirWyNwBZ2hFfZzWAnbk64qAk2N4g HVPBFOPUcfuu1OEXEsWigl+28eWebv+ysJNijSBNwcqFxnukh/9uxAYBe692Q+TsOri/ 3OttaM5eMYM+ZEXbzAiinqjKwDblohcxtR6A1qxAug1Kn5JlHT8wCz6ybgTm2xPwR911 oJAFcZyH/d2H4PxNn1myup9zhTLkaYV6DjqIavYv+CaRHU+6niAyOoLDtimARIsvggWh HycSEnWDzVyTZvu5/yJhO2QdeqpZT0by4twQew3X2/X21fTQ8as0DmNvKgnOZlMC52Lw GUyQ== X-Forwarded-Encrypted: i=1; AJvYcCW/p/TVnXdLeKiHFce2s/Epin4Wv+P/SK44fZwhRTIrSaakKLK33bYIkD16/bmmQOe8AzowySV6CrsAW76MVYJZusjkVw==@freebsd.org X-Gm-Message-State: AOJu0YwEPVh1Fzu8HDyDoo00tvmgdRmIE7xDzRtvfaoAAW/RLvwu2USk AL0VdxvH7MHK2PT5xAp5JBalK+9e/l6YsIFFSKw+snjk6mrrEnDPT86dd9p2Siy/ZxOn74z/z4L Wsy/4kVnsfrCPCWlw1e+gT0H4coONa/Px5eaQvw== X-Google-Smtp-Source: AGHT+IGhwpuJ1BwH4balOklYTP5kKDDeCBu4LeVwT6EGB9UCvLKDtwpJJu6JtKdeJtLQ1H1LBsZjXsZnIARVpoLhjgk= X-Received: by 2002:a05:6358:7183:b0:1b5:fb78:fb9b with SMTP id e5c5f4694b2df-1b7ef691e7bmr334408155d.14.1725119778631; Sat, 31 Aug 2024 08:56:18 -0700 (PDT) List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 References: <202408200902.47K92BsJ078096@gitrepo.freebsd.org> <77cbcd47-1778-4e71-b824-4e75e0e4b2d6@FreeBSD.org> <19DBDC11-EB0F-48C0-9AAE-9EED087F0AB6@freebsd.org> <8a9faa7c-1f81-4aa8-adac-a6f07d7d73ff@FreeBSD.org> In-Reply-To: <8a9faa7c-1f81-4aa8-adac-a6f07d7d73ff@FreeBSD.org> From: Warner Losh Date: Sat, 31 Aug 2024 09:56:07 -0600 Message-ID: Subject: Re: git: 43e8849bc294 - main - conf: Enable BTI checking in the arm64 kernel To: John Baldwin Cc: Andrew Turner , Jessica Clarke , "src-committers@freebsd.org" , "dev-commits-src-all@freebsd.org" , "dev-commits-src-main@freebsd.org" Content-Type: multipart/alternative; boundary="00000000000099ca480620fcbf30" X-Spamd-Bar: -- X-Spamd-Result: default: False [-3.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20230601.gappssmtp.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_DN_SOME(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; MISSING_XM_UA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::336:from]; MLMMJ_DEST(0.00)[dev-commits-src-main@freebsd.org]; RCPT_COUNT_FIVE(0.00)[6]; R_SPF_NA(0.00)[no SPF record]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[dev-commits-src-main@freebsd.org]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20230601.gappssmtp.com:+] X-Rspamd-Queue-Id: 4Wx04N3WgHz4HvJ --00000000000099ca480620fcbf30 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Aug 30, 2024 at 9:35=E2=80=AFAM John Baldwin wrot= e: > On 8/30/24 04:55, Andrew Turner wrote: > > > > > >> On 29 Aug 2024, at 17:02, Jessica Clarke wrote: > >> > >> On 21 Aug 2024, at 15:28, John Baldwin wrote: > >>> > >>> On 8/20/24 05:02, Andrew Turner wrote: > >>>> The branch main has been updated by andrew: > >>>> URL: > https://cgit.FreeBSD.org/src/commit/?id=3D43e8849bc29414036ccaef7788de95a= 07ad32ab5 > >>>> commit 43e8849bc29414036ccaef7788de95a07ad32ab5 > >>>> Author: Andrew Turner > >>>> AuthorDate: 2024-08-19 12:59:49 +0000 > >>>> Commit: Andrew Turner > >>>> CommitDate: 2024-08-20 08:49:15 +0000 > >>>> conf: Enable BTI checking in the arm64 kernel > >>>> To ensure new code has BTI support make it an error to not > have the > >>>> BTI ELF note when linking the kernel and kernel modules. > >>>> Reviewed by: kib, emaste > >>>> Sponsored by: Arm Ltd > >>>> Differential Revision: https://reviews.freebsd.org/D45469 > >>> > >>> This has broken two of the GitHub CI actions using clang 12 and clang > 13. > >>> Please fix this to be conditional on a supported linker version (or > perhaps > >>> add a new linker feature to bsd.linker.mk). > >> > >> This is still broken. If it=E2=80=99s not fixed promptly I will just r= evert > >> this change; we can=E2=80=99t leave CI broken, especially when it gets= used to > >> test GitHub PRs. Please stop breaking building with older toolchains, > >> this isn=E2=80=99t the first time it=E2=80=99s happened. > > > > See https://github.com/freebsd/freebsd-src/pull/1393 and > https://github.com/freebsd/freebsd-src/pull/1399 > > I do think we probably want to flesh out a bit more what kind of policy > we want for the range of compiler versions to support. E.g. in the GDB > project the policy is something like "will not use a version of C++ newer > than a compiler from 10 years ago" (IIRC). There they will also look bac= k > to whatever LTS distros are active and what is the newest compiler that > can be installed on those LTS via the equivalent of ports. > QEMU does similar things. > Traditionally we used to only support compiling FreeBSD N on FreeBSD N-1 > (though we have often supported older versions of FreeBSD). However, we > now also support cross-compiling from Linux and macOS, so we probably wan= t > to widen the support base a bit. I would say a first stab perhaps is tha= t > main and any supported stable and release branches should be buildable on > a host running any of those versions. That is, FreeBSD 13 is still > supported so we should keep main building on it directly without requirin= g > a jump to FreeBSD 14 first. Once 13 EOLs then we can stop supporting 13, > and that would gives folks on 13 a way to step up to the supported versio= n > of 14 at the time of the EOL. That said, presuambly fairly recent versio= ns > of LLVM (and GCC for that matter) will be available in ports for supporte= d > versions, so that doesn't necessarily require a very wide range of > supported > compiler versions. Supporting the "native" compiler on those releases > would be a nice goal that I think is probably achievable with minimal > effort. > Recent history has been such that all supported branches can jump to head. It's been more like N-2 or N-3 (12 -> head still works last I tried it, but it's likely to break at any random llvm import and we've had once since I (accidentally) tried it). In general, it's good policy if we can do it. In the earlier days of the project, there was considerably more API churn that required more compatibility shim= s than we need these days. It's usually just a matter of having a new enough C++ compiler for the latest language features used by llvm. I think the last breakage I looked into was like that (11 didn't have a new enough llvm for 14-current). As you note, we have ports/pkgs that can do that, but only if we keep the older recently not supported branches final packages around... Also to clarify one point: It's only the last supported release on the stable branch or newer that can jump to -current, not any point since the branch (though it often works, we dropped that effort some time ago when its main champion moved on from contributing to the FreeBSD project). I think stable-11 got a 'final' llvm update late in the branche's life because we hadn't had a release in too long, but I may be misremembering. But having said all that, I agree: this isn't going to widen things all that much. Today our oldest supported release is 14.0 which shipped with clang 16, and > 13.3 shipped with clang 17. Both of those are quite a bit newer than > clang 12 > that is current oldest version in CI. I have no idea what version range > might be sensible for Linux and macOS. Presumably we'd want to settle on > some range of ubuntu LTS versions and whatever is the newest version you > can > install on the oldest LTS as the minimum. > > clang 12 might have been used in 13.1 (not mentioned in the release notes= , > 13.0 used clang 11, 13.2 used clang 14) and 12.3 (12.4 used clang 13, > 12.2 used clang 10, 12.3 didn't mention it). > That gets back to the question of 'tip of the branch' or 'from the branch point'. I think that we should require only tip of branch (so we don't gate using new compiler features in head for something we shipped in 13.0), but encourage people to put the API compat shims into the build library for a wider range (since they will likely also be needed for Linux/MacOS too, independent of compiler version). > I know CHERI LLVM is still on LLVM 15 (though moving to LLVM 16 soon, but > Morello LLVM is 14 still I think). libc++ is also making this more > complicated as they are providing very little compatability at all. > They've > already ripped out support for GCC 14 I believe, so you can't build curre= nt > libc++ with a released version of GCC or something crazy. > Upstream will, perhaps sadly, drive a lot of our policy in this area. When upstream de-orbits support for something, it's hard for us to keep maintaining it and keep updating to upstream without some kind of compelling need. Though I recently spend a fair amount of time coming up with a fairly minimal tweak to our cdefs.h that makes it more compatible with the crazy things that libc++ does to still support compiling older C++ standards. Which brings up another possible issue. We don't support building FreeBSD, itself, with anything but a fairly up-to-date wrt standards compiler. However, we support building binaries from sources that compile with C89 and newer, and C++98 and newer (though really this is likely C++11 for most things, given the upstream de-orbiting of support for older standard binaries). We may need to carefully call this out as well... The older C++ standards don't matter too much (most of the C++ plorts actually want the newer C++), but there's still a lot of packages that compile with a strict C89 feature set that break in subtle ways when we break support for that... Warner --00000000000099ca480620fcbf30 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Fri, Aug 30, 2024 at 9:35=E2=80=AF= AM John Baldwin <jhb@freebsd.org&= gt; wrote:
On 8/= 30/24 04:55, Andrew Turner wrote:
>
>
>> On 29 Aug 2024, at 17:02, Jessica Clarke <jrtc27@freebsd.org> wrote:
>>
>> On 21 Aug 2024, at 15:28, John Baldwin <jhb@FreeBSD.org> wro= te:
>>>
>>> On 8/20/24 05:02, Andrew Turner wrote:
>>>> The branch main has been updated by andrew:
>>>> URL: https://cgit.FreeBSD.org/src/commit/?id=3D43e8849bc29414036ccaef7788de9= 5a07ad32ab5
>>>> commit 43e8849bc29414036ccaef7788de95a07ad32ab5
>>>> Author:=C2=A0 =C2=A0 =C2=A0Andrew Turner <andrew@FreeBS= D.org>
>>>> AuthorDate: 2024-08-19 12:59:49 +0000
>>>> Commit:=C2=A0 =C2=A0 =C2=A0Andrew Turner <andrew@FreeBS= D.org>
>>>> CommitDate: 2024-08-20 08:49:15 +0000
>>>>=C2=A0 =C2=A0 =C2=A0conf: Enable BTI checking in the arm64 = kernel
>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 To ensure new code has B= TI support make it an error to not have the
>>>>=C2=A0 =C2=A0 =C2=A0BTI ELF note when linking the kernel an= d kernel modules.
>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Reviewed by:=C2=A0 =C2= =A0 kib, emaste
>>>>=C2=A0 =C2=A0 =C2=A0Sponsored by:=C2=A0 =C2=A0Arm Ltd
>>>>=C2=A0 =C2=A0 =C2=A0Differential Revision:=C2=A0 h= ttps://reviews.freebsd.org/D45469
>>>
>>> This has broken two of the GitHub CI actions using clang 12 an= d clang 13.
>>> Please fix this to be conditional on a supported linker versio= n (or perhaps
>>> add a new linker feature to bsd.linker.mk).
>>
>> This is still broken. If it=E2=80=99s not fixed promptly I will ju= st revert
>> this change; we can=E2=80=99t leave CI broken, especially when it = gets used to
>> test GitHub PRs. Please stop breaking building with older toolchai= ns,
>> this isn=E2=80=99t the first time it=E2=80=99s happened.
>
> See https://github.com/freebsd/freebsd-src/pu= ll/1393 and https://github.com/freebsd/freebsd-s= rc/pull/1399

I do think we probably want to flesh out a bit more what kind of policy
we want for the range of compiler versions to support.=C2=A0 E.g. in the GD= B
project the policy is something like "will not use a version of C++ ne= wer
than a compiler from 10 years ago" (IIRC).=C2=A0 There they will also = look back
to whatever LTS distros are active and what is the newest compiler that
can be installed on those LTS via the equivalent of ports.
=

QEMU does similar things.
=C2=A0
Traditionally we used to only support compiling FreeBSD N on FreeBSD N-1 (though we have often supported older versions of FreeBSD).=C2=A0 However, = we
now also support cross-compiling from Linux and macOS, so we probably want<= br> to widen the support base a bit.=C2=A0 I would say a first stab perhaps is = that
main and any supported stable and release branches should be buildable on a host running any of those versions.=C2=A0 That is, FreeBSD 13 is still supported so we should keep main building on it directly without requiring<= br> a jump to FreeBSD 14 first.=C2=A0 Once 13 EOLs then we can stop supporting = 13,
and that would gives folks on 13 a way to step up to the supported version<= br> of 14 at the time of the EOL.=C2=A0 That said, presuambly fairly recent ver= sions
of LLVM (and GCC for that matter) will be available in ports for supported<= br> versions, so that doesn't necessarily require a very wide range of supp= orted
compiler versions.=C2=A0 Supporting the "native" compiler on thos= e releases
would be a nice goal that I think is probably achievable with minimal
effort.

Recent history has been such th= at all supported branches can jump to head. It's
been more li= ke N-2 or N-3 (12 -> head still works last I tried it, but it's like= ly to
break at any random llvm import and we've had once sinc= e I (accidentally) tried
it). In general, it's good policy if= we can do it. In the earlier days of the project,
there was cons= iderably more API churn that required more compatibility shims
th= an we need these days. It's usually just a matter of having a new enoug= h C++
compiler for the latest language features used by llvm. I t= hink the last breakage
I looked into was like that (11 didn't= have a new enough llvm for 14-current). As
you note, we have por= ts/pkgs that can do that, but only if we keep the older recently
= not supported branches final packages around...=C2=A0=C2=A0

<= /div>
Also to clarify one point: It's only the last supported relea= se on the stable branch or newer
that can jump to -current, not a= ny point since the branch (though it often works, we
dropped that= effort some time ago when its main champion moved on from contributing
to the FreeBSD project). I think stable-11 got a 'final' llv= m update late in the branche's
life because we hadn't had= a release in too long, but I may be misremembering.

But having said all that, I agree: this isn't going to widen things = all that much.

Today our oldest supported release is 14.0 which shipped with cl= ang 16, and
13.3 shipped with clang 17.=C2=A0 Both of those are quite a bit newer than = clang 12
that is current oldest version in CI.=C2=A0 I have no idea what version ran= ge
might be sensible for Linux and macOS.=C2=A0 Presumably we'd want to se= ttle on
some range of ubuntu LTS versions and whatever is the newest version you ca= n
install on the oldest LTS as the minimum.

clang 12 might have been used in 13.1 (not mentioned in the release notes,<= br> 13.0 used clang 11, 13.2 used clang 14) and 12.3 (12.4 used clang 13,
12.2 used clang 10, 12.3 didn't mention it).

<= /div>
That gets back to the question of 'tip of the branch' or = 'from the branch point'. I think
that we should require o= nly tip of branch (so we don't gate using new compiler features
in head for something we shipped in 13.0), but encourage people to put t= he API compat
shims into the build library for a wider range (sin= ce they will likely also be needed for
Linux/MacOS too, independe= nt of compiler version).
=C2=A0
I know CHERI LLVM is still on LLVM 15 (though moving to LLVM 16 soon, but Morello LLVM is 14 still I think).=C2=A0 libc++ is also making this more complicated as they are providing very little compatability at all.=C2=A0 T= hey've
already ripped out support for GCC 14 I believe, so you can't build cur= rent
libc++ with a released version of GCC or something crazy.
<= div>
Upstream will, perhaps sadly, drive a lot of our policy = in this area. When upstream de-orbits
support for something, it&#= 39;s hard for us to keep maintaining it and keep updating to upstream
=
without some kind of compelling need. Though I recently spend a fair a= mount of time coming
up with a fairly minimal tweak to our cdefs.= h that makes it more compatible with the crazy things
that libc++= does to still support compiling older C++ standards.

<= div>Which brings up another possible issue. We don't support building F= reeBSD, itself, with anything
but a fairly up-to-date wrt standar= ds compiler. However, we support building binaries from sources
t= hat compile with C89 and newer, and C++98 and newer (though really this is = likely C++11 for most
things, given the upstream de-orbiting of s= upport for older standard binaries). We may need to carefully
cal= l this out as well... The older C++ standards don't matter too much (mo= st of the C++ plorts actually
want the newer C++), but there'= s still a lot of packages that compile with a strict C89 feature set
<= div>that break in subtle ways when we break support for that...
<= br>
Warner=C2=A0
--00000000000099ca480620fcbf30--