Porting S3QL and ca-root-nss.crt: Python unable to find needed certificates
Kubilay Kocak
koobs at FreeBSD.org
Sat Oct 3 14:52:57 UTC 2015
On 3/10/2015 11:20 PM, Niklaas Baudet von Gersdorff wrote:
> Hi,
>
> I'm porting S3QL, see https://bitbucket.org/nikratio/s3ql/overview. It
> creates a mountable filesystem on a Amazon S3 bucket. I already came
> this far: https://github.com/niklaas/freebsd-port-s3ql which also
> requires two python modules there are so far no ports for too:
>
> llfuse: https://github.com/niklaas/freebsd-port-llfuse
>
> dugong: https://github.com/niklaas/freebsd-port-dugong
>
> `poudriere testport` works fine. I installed the port on a FreeBSD
> system successfully and it seems to work fine. S3QL connects to the S3
> bucket via SSL. But the connection only works if I use the following
> command, explicitly stating the location of ca-root-nss.crt:
>
> mkfs.s3ql --backend-options
> ssl-ca-path=/usr/local/share/certs/ca-root-nss.crt s3://<bucket-name>
>
> Not doing so causes the following errors:
>
>> Traceback (most recent call last):
>> File "/usr/local/bin/mount.s3ql", line 9, in <module>
>> load_entry_point('s3ql==2.15', 'console_scripts', 'mount.s3ql')()
>> File "/usr/local/lib/python3.4/site-packages/s3ql/mount.py", line 120, in main
>> options.authfile, options.compress)
>> File "/usr/local/lib/python3.4/site-packages/s3ql/common.py", line 340, in get_backend_factory
>> backend.fetch('s3ql_passphrase')
>> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/common.py", line 351, in fetch
>> return self.perform_read(do_read, key)
>> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/common.py", line 107, in wrapped
>> return method(*a, **kw)
>> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/common.py", line 314, in perform_read
>> fh = self.open_read(key)
>> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/common.py", line 107, in wrapped
>> return method(*a, **kw)
>> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/s3c.py", line 302, in open_read
>> resp = self._do_request('GET', '/%s%s' % (self.prefix, key))
>> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/s3c.py", line 437, in _do_request
>> query_string=query_string, body=body)
>> File "/usr/local/lib/python3.4/site-packages/s3ql/backends/s3c.py", line 668, in _send_request
>> self.conn.send_request(method, path, body=body, headers=headers)
>> File "/usr/local/lib/python3.4/site-packages/dugong/__init__.py", line 508, in send_request
>> self.timeout)
>> File "/usr/local/lib/python3.4/site-packages/dugong/__init__.py", line 1396, in eval_coroutine
>> if not next(crt).poll(timeout=timeout):
>> File "/usr/local/lib/python3.4/site-packages/dugong/__init__.py", line 535, in co_send_request
>> self.connect()
>> File "/usr/local/lib/python3.4/site-packages/dugong/__init__.py", line 444, in connect
>> self._sock = self.ssl_context.wrap_socket(self._sock, server_hostname=server_hostname)
>> File "/usr/local/lib/python3.4/ssl.py", line 365, in wrap_socket
>> _context=self)
>> File "/usr/local/lib/python3.4/ssl.py", line 583, in __init__
>> self.do_handshake()
>> File "/usr/local/lib/python3.4/ssl.py", line 810, in do_handshake
>> self._sslobj.do_handshake()
>> ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
>
> I did some research and found this *fixed* bug which is more or less recent:
>
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=196431
>
> I was wondering whether there are still some issues with
> security/ca_root_nss or whether I forgot to specify something in the
> Makefile?
Hi Niklaas,
I'm not aware of any reported issues. You could confirm this by reducing
the test case to exclude your port, and attempting to test python/ssl
functionality alone with something like:
import socket, ssl
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
context.verify_mode = ssl.CERT_REQUIRED
context.check_hostname = True
context.load_default_certs()
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ssl_sock = context.wrap_socket(s, server_hostname='www.verisign.com')
ssl_sock.connect(('www.verisign.com', 443))
This works for me (no errors) on my local Python 3.4 (from ports)
installation.
> How can I find out where python is looking for the certificates? This
> would enable me to create a symlink which could be added to the final
> version of my port of net/s3ql then too. (Note: I don't know python.)
As per the commit messages for the bugzilla issue you mentioned, python,
unlike other software (like curl iirc) uses an OpenSSL function, *not*
an internal list of paths, to derive CA certificate path locations. See
this commit message for full details:
https://svnweb.freebsd.org/ports?view=revision&revision=378720
Note: The path locations used by this function are determined at OpenSSL
*build time*. The changes made in:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=196431 and
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=189811
ensured that symlinks were created so that the certs could be found at
those build-time path locations.
If you're not using OpenSSL from *ports* (ie, from base) then you'll
want to ensure the ETCSYMLINK option is enabled when installing
ca_root_nss from ports.
Note: The package for ca_root_nss has this option enabled by default,
which is the exact and only change bug #189811 fixed.
Jump in #freebsd-python on IRC (freenode) if your investigations turn up
any issues regarding Python/SSL certificate verification.
./koobs
More information about the freebsd-ports
mailing list