maintainer-feedback requested: [Bug 251562] lang/python37: SSLError 'certificate verify failed' despite correct looking /etc/ssl/cert.pem

[bugzilla-noreply at freebsd.org]

Bugzilla Automation <bugzilla at FreeBSD.org> has asked freebsd-python (Nobody)
<python at FreeBSD.org> for maintainer-feedback:
Bug 251562: lang/python37: SSLError 'certificate verify failed' despite correct
looking /etc/ssl/cert.pem
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=251562



--- Description ---
Overview:
the command 'python3.7 -c "import requests;
print(requests.get('https://server:8443').text)"' fails with a 'certificate
verify failed' error when additional root certificates from the trust store
should be used

Steps to Reproduce:
1) import a self-signed root certificate into /usr/share/certs/trusted, i.e.
use: cp myroot.pem /usr/share/certs/trusted)
2) get fingerprint from root certificate and add softlink to /etc/ssl/certs,
i.e. use: ln -s /usr/share/certs/trusted/myroot.pem /etc/ssl/certs/97efb5b5.0
3) append content of myroot.pem to /etc/ssl/cert.pem, i.e. use: cat myroot.pem
| sudo tee -a /etc/ssl/cert.pem)
4) verify that OpenSSL can verify a server certificate that is derived from the
newly imported root certificate, i.e. use: openssl s_client -connect
server:8443
5) make sure that python is looking into the correct directories for SSL CERTS,
i.e. use: python3.7 -c "import ssl; print(ssl.get_default_verify_paths())"
5) try a python request to the server, i.e. use: python3.7 -c "import requests;
print(requests.get('https://server:8443').text)"

Actual Results:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py",
line 488, in wrap_socket
    cnx.do_handshake()
  File "/usr/local/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1915, in
do_handshake
    self._raise_ssl_error(self._ssl, result)
  File "/usr/local/lib/python3.7/site-packages/OpenSSL/SSL.py", line 1647, in
_raise_ssl_error
    _raise_current_error()
  File "/usr/local/lib/python3.7/site-packages/OpenSSL/_util.py", line 54, in
exception_from_error_queue
    raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'tls_process_server_certificate',
'certificate verify failed')]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line
677, in urlopen
    chunked=chunked,
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line
381, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line
978, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line
371, in connect
    ssl_context=context,
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 386,
in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py",
line 494, in wrap_socket
    raise ssl.SSLError("bad handshake: %r" % e)
ssl.SSLError: ("bad handshake: Error([('SSL routines',
'tls_process_server_certificate', 'certificate verify failed')])",)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 449,
in send
    timeout=timeout
  File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line
727, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line
446, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='server',
port=8443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad
handshake: Error([('SSL routines', 'tls_process_server_certificate',
'certificate verify failed')])")))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 75, in
get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 60, in
request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 533,
in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 646,
in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 514,
in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='server', port=8443):
Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake:
Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify
failed')])")))

Expected Results:
HTML content from the server's website (i.e. 'Hello World')

Additional Information:
When I modify the request so I explicitly tell python to use /etc/ssl/cert.pem
for verification, I get my 'Hello World' and no error...

python3.7 -c "import requests; print(requests.get('https://server:8443',
verify='/etc/ssl/cert.pem').text)"

Also, the output of python's ssl.get_default_verify_paths() shows me that it
seems to look all in the right places:

DefaultVerifyPaths(cafile='/etc/ssl/cert.pem', capath='/etc/ssl/certs',
openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/etc/ssl/cert.pem',
openssl_capath_env='SSL_CERT_DIR', openssl_capath='/etc/ssl/certs')