From nobody Wed May 03 23:36:24 2023 X-Original-To: freebsd-arch@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QBYJ76kVLz49PKd for ; Wed, 3 May 2023 23:36:55 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [IPv6:2a00:1450:4864:20::530]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QBYJ750xPz49GG for ; Wed, 3 May 2023 23:36:55 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x530.google.com with SMTP id 4fb4d7f45d1cf-50bc25f0c7dso9347660a12.3 for ; Wed, 03 May 2023 16:36:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20221208.gappssmtp.com; s=20221208; t=1683157014; x=1685749014; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=NAlIJuRha5kD9m55hw+fP2Ghez/hk47NeA2LtQnFhBU=; b=XsHnUNm+lFiX75SEawbTwJ5y03gkL6MPZFxawL4GscN+M8BSLHCdTMnbc/xM5PA8G1 TNJBTlQbBfUIxWKGZW9GbrhGWDz15sE6jYaJ5bco78zwsrNUQyyEsNQeqt6O1FioPuuW DU1yB8VJ2FMPVNxP3odOGb7+3Eg7BC5yXzwEXeGSCXixtT6jpNzTyiHwTnlHlTmYZYt2 MX2fjVDIEeP4CDOuvuCpr4HM3B7ma7OCPsKQ6vRZmbc5ehQjPV/mTmnpYhGC/cgF3b20 uGOM66awNNy9fsphdus9lEF7tuj2yZ03knacFHJwu+hEMst6ckCuOcMzDAFK7qdvdJ9t Ennw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683157014; x=1685749014; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=NAlIJuRha5kD9m55hw+fP2Ghez/hk47NeA2LtQnFhBU=; b=PaUeJ4NYCq2mlHR1EBqWBrHyXzDSzq827aupd4OTsR0gHaH8asYpPmTxxLz3YERpWC vzsbGfHBXPQwK6hFG+Z9tB2SURv1TxgpsNXOzam6245KxSO8de6tFjORJORPsik7JW4L TUx6tD03RtktCYGw07LqgIKZWgQpvmAxYwfOy9jt+hZnavUqYWj1cHp8uqe1oIkw4CFW GFZ9U2CSwXqmeqNHfG01KC327Q+uRPkS6FbH2j4rDI4NWzGv3efh38jHLVRbEYyt1mH6 GBpkJBhUBvCkOcA+a1xvuMY/nog7TJj3m50tEAjssL6T/gp9Kg8lS+VOAys4GOvK4O8U 3DjQ== X-Gm-Message-State: AC+VfDzSTZpsnHdTHoSyGFvpQWnKA++lXbW6FF8wOdNPQ0s4sCrN1UKg SoDWRnnTg12vKoCUxaTGYy6CrIlvHeMsMv1zi46XoWpsy1jzR8OH X-Google-Smtp-Source: ACHHUZ6u9RMjU+iAqOLNXFcESYhOhJXInYFCidn+4GrdM4Wi+PyxEAIK1LFe5KAck5Eo9ffigCxODh8KiT/EeUfPF+A= X-Received: by 2002:a17:907:ea2:b0:953:1fba:7803 with SMTP id ho34-20020a1709070ea200b009531fba7803mr5829794ejc.15.1683157013650; Wed, 03 May 2023 16:36:53 -0700 (PDT) List-Id: Discussion related to FreeBSD architecture List-Archive: https://lists.freebsd.org/archives/freebsd-arch List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arch@freebsd.org MIME-Version: 1.0 References: <12f8559c-d696-5344-98d5-1751d04088af@FreeBSD.org> In-Reply-To: From: Warner Losh Date: Wed, 3 May 2023 17:36:24 -0600 Message-ID: Subject: Re: OpenSSL 3.0 for 14.0-RELEASE: issues with 1.x/3.x symbol clashing, ports linking against base OpenSSL, ports that don't compile/link against OpenSSL 3, etc To: Pierre Pronchery Cc: "freebsd-arch@freebsd.org" Content-Type: multipart/alternative; boundary="000000000000e673af05fad287be" X-Rspamd-Queue-Id: 4QBYJ750xPz49GG X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --000000000000e673af05fad287be Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, May 3, 2023, 5:10 PM Pierre Pronchery wrote: > Hi everyone, > > On 5/2/23 23:24, John Baldwin wrote: > > On 5/2/23 2:59 AM, Antoine Brodin wrote: > >> On Tue, May 2, 2023 at 1:55=E2=80=AFAM Enji Cooper > wrote: > >>> > >>> Hello, > >>> One of the must-haves for 14.0-RELEASE is the introduction of OpenSSL > >>> 3.0 into the base system. This is a must because, in short, OpenSSL > >>> 1.1 is no longer supported as of 09/26/2023 [1]. > >>> > >>> I am proposing OpenSSL be made private along with all dependent > >>> libraries, for the following reasons: > >>> 1. More than a handful of core ports, e.g., security/py-cryptography > >>> [2] [3], still do not support OpenSSL 3.0. > >>> i. If other dependent ports (like lang/python38, etc) move to OpenSSL > >>> 3, the distributed modules would break on load due to clashing > >>> symbols if the right mix of modules were dlopen=E2=80=99ed in a speci= fic > >>> order (importing ssl, then importing hazmat=E2=80=99s crypto would fa= il). > >>> ii. Such ports should be deprecated/marked broken as I=E2=80=99ve rec= ommended > >>> on the 3.0 exp-run PR [4]. > >>> 2. OpenSSL 1.1 and 3.0 have clashing symbols, which makes linking in > >>> both libraries at runtime impossible without resorting to a number of > >>> linker tricks hiding the namespaces using symbol prefixing of public > >>> symbols, etc. > >>> > >>> The libraries which would need to be made private are as follows: > >>> - kerberos > >>> - libarchive > >>> - libbsnmp > >>> - libfetch [5] > >>> - libgeli > >>> - libldns > >>> - libmp > >>> - libradius > >>> - libunbound > >> > >> In my opinion this is a huge amount of work a few weeks before the > >> release. Focusing on updating OpenSSL and those core ports may be > >> simpler. > > > > This is my view. I think making OpenSSL private is a very huge task, a= nd > > fraught with peril in ways that haven't been thought about yet (e.g. PA= M) > > and that we can't hold up OpenSSL 3 while we wait for this. Instead, I > > think > > we need to be moving forward with OpenSSL 3 in base as-is. We will hav= e > to > > fix ports to work with OpenSSL 3 regardless (though this does make that > > pain > > in ports happen sooner). Moving libraries private can happen > orthogonally > > with getting base to work with OpensSL 3. > > I have started to look at updating OpenSSL to version 3.0.8 in base, > using the existing vendor/openssl-3.0 branch. > > My progress can be found at > https://github.com/khorben/freebsd-src/tree/khorben/openssl-3.0. I > regularly force-push to keep a consistent and nice commit history, > before possibly applying for a merge. > > So far the status is: > > - libssl, libcrypto build on amd64, i386, less sure about aarch64, other > architectures not tested > - libfetch builds, uses libmd in addition to OpenSSL > - libradius builds, same thing > - libarchive builds > - libunbound builds, but not unbound > - libmp builds > > I used libmd to reach a buildable status faster, since the equivalent > MD5_*() API is now deprecated in OpenSSL 3. If MD5 is still allowed in > OpenSSL 3, we can avoid the dependency on libmd again. (anyone got > sample code for this?) > > Meanwhile I keep trying to build the rest of the system, hopefully in > time for a possible inclusion in -14. > > Reviews and tests on the whole thing will be more than welcome in any cas= e! > Cool. Id like to echo the namespace remapping suggestion. OpenSSL tends to be basically a leaf requirement. If we always remapping our in tree openssl, then if someone uses, say, libfetch and something else that requires openssl1, they can coexist in the same binary. Without the namespace remapping, we get a guaranteed conflict. It would increase our flexibility. And it could potentially be mfc'd if the old openssl were left as an optional component for people on stable stuck with it as a requirement. It would mitigate the EOL issues while giving an escape hatch that's not insane (modulo the elevated risk) I'll have to take a closer look at the branch. Warner Cheers & HTH, > -- > Pierre Pronchery > > > --000000000000e673af05fad287be Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Wed, May 3, 2023, 5:10 PM Pierre Pronchery <pierre@freebsdfoundation.org> wrote:
=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Hi everyone,

On 5/2/23 23:24, John Baldwin wrote:
> On 5/2/23 2:59 AM, Antoine Brodin wrote:
>> On Tue, May 2, 2023 at 1:55=E2=80=AFAM Enji Cooper <yaneurab= eya@gmail.com> wrote:
>>>
>>> Hello,
>>> One of the must-haves for 14.0-RELEASE is the introduction of = OpenSSL
>>> 3.0 into the base system. This is a must because, in short, Op= enSSL
>>> 1.1 is no longer supported as of 09/26/2023 [1].
>>>
>>> I am proposing OpenSSL be made private along with all dependen= t
>>> libraries, for the following reasons:
>>> 1. More than a handful of core ports, e.g., security/py-crypto= graphy
>>> [2] [3], still do not support OpenSSL 3.0.
>>> i. If other dependent ports (like lang/python38, etc) move to = OpenSSL
>>> 3, the distributed modules would break on load due to clashing=
>>> symbols if the right mix of modules were dlopen=E2=80=99ed in = a specific
>>> order (importing ssl, then importing hazmat=E2=80=99s crypto w= ould fail).
>>> ii. Such ports should be deprecated/marked broken as I=E2=80= =99ve recommended
>>> on the 3.0 exp-run PR [4].
>>> 2. OpenSSL 1.1 and 3.0 have clashing symbols, which makes link= ing in
>>> both libraries at runtime impossible without resorting to a nu= mber of
>>> linker tricks hiding the namespaces using symbol prefixing of = public
>>> symbols, etc.
>>>
>>> The libraries which would need to be made private are as follo= ws:
>>> - kerberos
>>> - libarchive
>>> - libbsnmp
>>> - libfetch [5]
>>> - libgeli
>>> - libldns
>>> - libmp
>>> - libradius
>>> - libunbound
>>
>> In my opinion this is a huge amount of work a few weeks before the=
>> release.=C2=A0 Focusing on updating OpenSSL and those core ports m= ay be
>> simpler.
>
> This is my view.=C2=A0 I think making OpenSSL private is a very huge t= ask, and
> fraught with peril in ways that haven't been thought about yet (e.= g. PAM)
> and that we can't hold up OpenSSL 3 while we wait for this.=C2=A0 = Instead, I
> think
> we need to be moving forward with OpenSSL 3 in base as-is.=C2=A0 We wi= ll have to
> fix ports to work with OpenSSL 3 regardless (though this does make tha= t
> pain
> in ports happen sooner).=C2=A0 Moving libraries private can happen ort= hogonally
> with getting base to work with OpensSL 3.

I have started to look at updating OpenSSL to version 3.0.8 in base,
using the existing vendor/openssl-3.0 branch.

My progress can be found at
https://github.com/khorben= /freebsd-src/tree/khorben/openssl-3.0. I
regularly force-push to keep a consistent and nice commit history,
before possibly applying for a merge.

So far the status is:

- libssl, libcrypto build on amd64, i386, less sure about aarch64, other architectures not tested
- libfetch builds, uses libmd in addition to OpenSSL
- libradius builds, same thing
- libarchive builds
- libunbound builds, but not unbound
- libmp builds

I used libmd to reach a buildable status faster, since the equivalent
MD5_*() API is now deprecated in OpenSSL 3. If MD5 is still allowed in
OpenSSL 3, we can avoid the dependency on libmd again. (anyone got
sample code for this?)

Meanwhile I keep trying to build the rest of the system, hopefully in
time for a possible inclusion in -14.

Reviews and tests on the whole thing will be more than welcome in any case!=

= Cool. Id like to echo the namespace remapping suggestion.=C2=A0 OpenSSL ten= ds to be basically a leaf requirement.=C2=A0 If we always remapping our in = tree openssl, then if someone uses, say, libfetch and something else that r= equires openssl1, they can coexist in the same binary. Without the namespac= e remapping, we get a guaranteed conflict.

It would increase our flexibility. And it could potentia= lly be mfc'd if the old openssl were left as an optional component for = people on stable stuck with it as a requirement. It would mitigate the EOL = issues while giving an escape hatch that's not insane (modulo the eleva= ted risk)

I'll have = to take a closer look at the branch.=C2=A0

Warner

Cheers & HTH,
--
Pierre Pronchery


--000000000000e673af05fad287be--