Re: pkg_https:// failures related to, for example, "SSL certificate problem: certificate is not yet valid"

On 2024-07-04 09:51:53 (+0800), Mark Millard wrote:
> On Jul 3, 2024, at 17:47, Philip Paeps <philip@freebsd.org> wrote:
>> On 2024-07-04 01:27:03 (+0800), Mark Millard wrote:
>>> Bootstrapping pkg from 
>>> pkg+https://pkg.FreeBSD.org/FreeBSD:14:aarch64/quarterly, please 
>>> wait...
>>> Certificate verification failed for /CN=pkg.freebsd.org
>>> 0020616CE1680000:error:0A000086:SSL 
>>> routines:tls_post_process_server_certificate:certificate verify 
>>> failed:/usr/src/crypto/openssl/ssl/statem/statem_clnt.c:1890:
>>
>> As far as I can tell, at the time of this writing, all fifteen 
>> pkg.freebsd.org sites have the same certificate, and OpenSSL is happy 
>> with it.
>>
>>> Note the "pkg+https://".
>>>
>>> I had separate problems yesterday that I side stepped by
>>> testing use of just "pkg+http://", which worked. See:
>>
>> Use pkg+http.  This is the default.
>
> Hmm:
>
> # grep http /usr/src/usr.sbin/pkg/FreeBSD.conf.*
> /usr/src/usr.sbin/pkg/FreeBSD.conf.latest:  url: 
> "pkg+https://pkg.FreeBSD.org/${ABI}/latest",
> /usr/src/usr.sbin/pkg/FreeBSD.conf.quarterly:  url: 
> "pkg+https://pkg.FreeBSD.org/${ABI}/quarterly",
>
> Releases, snapshots, pkgbase, and artifacts all explicitly
> end up with https in /etc/pkg/FreeBSD.conf

Sorry.  The default does seem to have changed to HTTPS since I last 
looked.  The commit log suggests it was done only because it is now 
possible.  I don't think it's a good idea.  It only adds work (and work 
is heat) for no benefit.

>>> pkg with -d for the https context had its debug output
>>> reporting:
>>>
>>> * SSL certificate problem: certificate is not yet valid
>>
>> Does the system being bootstrapped have a real-time clock?  Common 
>> causes for this error are clocks set to 1970-01-01 or 2000-01-01.
>
> /var/log/messages confirms the time issue for my example
> boots that had the problem: it stayed back at Mar 16, not
> updating via ntpd as it normally does. (That date is
> probably from UFS. The system had not been booted since
> back then.)

That's what I suspected.

And this is another reason why HTTPS is a terrible default for pkg.  I 
don't think we should require a system to keep (reasonably) accurate 
time in order to be able to download packages.

> It does seem that /etc/pkg/FreeBSD.conf should avoid
> the https notation so that it presents an appropriate
> default.

I agree.  Adding bapt@ to Cc:.  I think this needs to be reverted.  It 
should probably also be an errata candidate so folks running releases 
can update packages even when their clocks get out of sync.

In addition to needlessly generating heat, pkg+https reduces the overall 
security of the system by making it more difficult for some 
installations to receive updates.

For the avoidance of doubt: I completely support HTTPS as a default for 
web traffic.  Privacy is important.  But pkg downloads are not web 
traffic.

Philip