Re: OpenSSL 3.0 for 14.0-RELEASE: issues with 1.x/3.x symbol clashing, ports linking against base OpenSSL, ports that don't compile/link against OpenSSL 3, etc

> On May 3, 2023, at 3:14 AM, Moin Rahman <bofh@freebsd.org> wrote:
> 
> 
> 
>> On May 2, 2023, at 3:55 AM, Enji Cooper <yaneurabeya@gmail.com> wrote:
>> 
>> Hello,
>> 	One of the must-haves for 14.0-RELEASE is the introduction of OpenSSL 3.0 into the base system. This is a must because, in short, OpenSSL 1.1 is no longer supported as of 09/26/2023 [1].
>> 
>> 	I am proposing OpenSSL be made private along with all dependent libraries, for the following reasons:
>> 	1. More than a handful of core ports, e.g., security/py-cryptography [2] [3], still do not support OpenSSL 3.0.
>> 		i. If other dependent ports (like lang/python38, etc) move to OpenSSL 3, the distributed modules would break on load due to clashing symbols if the right mix of modules were dlopen’ed in a specific order (importing ssl, then importing hazmat’s crypto would fail).
>> 		ii. Such ports should be deprecated/marked broken as I’ve recommended on the 3.0 exp-run PR [4].
>> 	2. OpenSSL 1.1 and 3.0 have clashing symbols, which makes linking in both libraries at runtime impossible without resorting to a number of linker tricks hiding the namespaces using symbol prefixing of public symbols, etc.
>> 
>> 	The libraries which would need to be made private are as follows:
>> 	- kerberos
>> 	- libarchive
>> 	- libbsnmp
>> 	- libfetch [5]
>> 	- libgeli
>> 	- libldns
>> 	- libmp
>> 	- libradius
>> 	- libunbound
>> 
>> 	I realize I’m jumping to a prescribed solution without additional discussion, but I’ve been doing offline analysis related to uplifting code from OpenSSL 1.x to 3.x over the last several months and this is the general prescribed solution I’ve come to which is needed for $work. My perspective might have some blind spots and some of the discussion done over IRC and might need to be rehashed here for historical reference/to widen the discussion for alternate solutions that don’t have the degree of tunnel vision which the solution I’m employing at $work requires.
>> 	I’ve tried to include some of the previously involved parties so they can chime in.
>> Thank you,
>> -Enji
>> 
>> 1. https://www.openssl.org/blog/blog/2023/03/28/1.1.1-EOL/
>> 2. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254853 .
>> 3. The reason why it hasn’t been upgraded is because newer versions require rustc to build, which apparently doesn’t work on QEMU builders due to missing emulation support: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254853 .
>> 4. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258413#c15
>> 5. If I remember correctly, some folks suggested that making libfetch private wasn’t required since the only port that required it was ports-mgmt/pkg, but I haven’t validated this claim.
> 
> Hi Enji,
> 
> I appreciate your work creating the bugs but please hold on a moment before you create the bugs. It will slow me down.
> 
> While you were wasting your time creating the ticket for nrpe3 I have already updated the port to 4.1.0 to unbreak. So until I have the final list which you will have by end of this week please do not create tickets.
> 
> And I have not exactly described the process too what I was doing. The list you are getting in my poudriere might have two possible failure reason. OpenSSL 3 or LLVM15; and some might be fixed with little intervention and testing. And as it's not possible to ask poudriere not to try BROKEN ports so I have marked some port as blacklisted which are unfixable or broken for other reasons. If you really would like to create tickets and chase upstream please do:
> find /usr/local/poudriere/ports/default -name Makefile -type f -d 3 -exec grep -E '(BROKEN_SSL\=|IGNORE_SSL\=).*openssl3' {} \+
> 
> Thanks for your cooperation.

Hi,

I have completed my final partial exp-run with ports that has USES=ssl in any form and the result is available from here:
https://pkg.bofh.network/build.html?mastername=MAIN-default-openssl3&build=2023-05-04_16h25m58s

You will see some blacklisted ports which fails to build on all supported RELEASE or HEAD. Apart from that I have tried to fix up as much as possible ports by updating/changing. I will not run anymore exp-run until we have a proper OSVERSION and OpenSSL 3.0.0 vendor branch merged into base and fix those ports that has BROKEN_SSL=openssl30 to conditionally mark BROKEN with ssl=base.

If someone wants to chase the upstreams of the ports feel free to do so.

During fixing these issues I had to upgrade some of the ports with portmgr(blanket) approval and in case if you think that I have overstepped on your plannings I apologies for that. However desperate time needs desperate measures.

Now I will go backup fixing LLVM15 issues with 14.0-RELEASE and ports.

Kind regards,
Moin