1323439 - Need functioning pyOpenSSL on testers

Reporter

Description

•

8 years ago

In order to test bug 1056934, our test TURN server needs to be able to serve TLS. Twisted (which this test TURN server uses) needs pyOpenSSL to do this. The copy of pyOpenSSL we have on http://pypi.pub.build.mozilla.org/pub does not appear to be functional on windows, linux, or OS X:


Windows:

16:46:04     INFO -    Running setup.py (path:C:\slave\test\build\venv\build\pyOpenSSL\setup.py) egg_info for package pyOpenSSL
16:46:05     INFO -      error: could not find 'ssleay32.dll'
16:46:05     INFO -      Complete output from command python setup.py egg_info:
16:46:05     INFO -      running egg_info
16:46:05     INFO -  creating pip-egg-info\pyOpenSSL.egg-info
16:46:05     INFO -  writing pip-egg-info\pyOpenSSL.egg-info\PKG-INFO
16:46:05     INFO -  writing top-level names to pip-egg-info\pyOpenSSL.egg-info\top_level.txt
16:46:05     INFO -  writing dependency_links to pip-egg-info\pyOpenSSL.egg-info\dependency_links.txt
16:46:05     INFO -  writing manifest file 'pip-egg-info\pyOpenSSL.egg-info\SOURCES.txt'
16:46:05     INFO -  warning: manifest_maker: standard file '-c' not found
16:46:05     INFO -  error: could not find 'ssleay32.dll'
16:46:05     INFO -  ----------------------------------------
16:46:05     INFO -  Cleaning up...

Linux:

15:18:49     INFO - ICE Server:   File "iceserver/iceserver.py", line 17, in <module>
15:18:49     INFO - ICE Server:     
15:18:49     INFO - ICE Server: from twisted.internet import reactor, protocol, ssl
15:18:49     INFO - ICE Server:   File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/twisted/internet/ssl.py", line 46, in <module>
15:18:49     INFO - ICE Server:     
15:18:49     INFO - ICE Server: from OpenSSL import SSL
15:18:49     INFO - ICE Server:   File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/OpenSSL/__init__.py", line 11, in <module>
15:18:49     INFO - ICE Server:     
15:18:49     INFO - ICE Server: import rand, crypto, SSL, tsafe
15:18:49     INFO - ICE Server: ImportError
15:18:49     INFO - ICE Server: : 
15:18:49     INFO - ICE Server: /builds/slave/test/build/venv/local/lib/python2.7/site-packages/OpenSSL/SSL.so: undefined symbol: SSLv2_method
15:18:49     INFO - ICE Server: 
15:18:49     INFO - ICE Server: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ProcessTerminated'>: A process has ended with a probable error condition: process ended with exit code 1.
15:18:49     INFO - ICE Server: ]
15:18:49     INFO - ICE Server websocket closed


OS X:

16:24:28     INFO - ICE Server:   File "iceserver/iceserver.py", line 764, in <module>
16:24:28     INFO - ICE Server:     tls_context_factory = ssl.DefaultOpenSSLContextFactory(KEY_FILE, CERT_FILE, SSL.TLSv1_2_METHOD)
16:24:28     INFO - ICE Server: AttributeError: 'module' object has no attribute 'TLSv1_2_METHOD'
16:24:28     INFO - ICE Server websocket closed

Byron Campen [:bwc]

Reporter

Updated

•

8 years ago

Summary: Need newer pyOpenSSL on builders → Need functioning pyOpenSSL on testers

Nils Ohlmeier [:drno]

Comment 1

•

8 years ago

I actually see two version of pyOpenSSL in http://pypi.pub.build.mozilla.org/pub:
  pyOpenSSL-0.10.tar.gz
  pyOpenSSL-16.2.0.tar.gz

The 0.10 version is from 2009, so that would not be surprising if that does not work.

:catlee pointed out to me that we should at least be able to make progress on this for Linux by modifying the Docker scripts for the Linux test machines. And the docker scripts appear to be this:
http://searchfox.org/mozilla-central/source/taskcluster/docker/desktop-test or http://searchfox.org/mozilla-central/source/taskcluster/docker/desktop1604-test

Not sure how pipi and the Docker things interact here though.

Greg Arndt [:garndt]

Comment 2

•

8 years ago

Those are the right directories to look at the dockerfiles for both images.  These dockerfiles make use of some in-tree magic that can grab files from a directory other than the working image directory and import them into the image.

Such as here:
https://dxr.mozilla.org/mozilla-central/source/taskcluster/docker/desktop1604-test/Dockerfile#26

Both images have a setup shell script that they run that configures system packages, etc as can be seen here [1].  In this example peep is used to install python packages.  You can find the setup script for desktop1604-test here [2].

If you change these any files that make up the build for the image, when you push to a branch, such as try, the images will be rebuilt with those changes and used for the tasks scheduled for that push.  That's an easy way to try your changes out prior to getting review and pushing elsewhere.

[1] https://dxr.mozilla.org/mozilla-central/source/taskcluster/docker/recipes/ubuntu1204-test-system-setup.sh#149
[2] https://dxr.mozilla.org/mozilla-central/source/taskcluster/docker/recipes/ubuntu1604-test-system-setup.sh

Nobody; OK to take it and work on it

Assignee

Updated

•

6 years ago

Component: General Automation → General

:dhouse

Comment 3

•

5 years ago

confirmed pyopenssl-19.1.0 includes openssl for macos (the wheels includes all but libc). So we do not expect problems updating on the macos workers:

[dhouse@t-mojave-r7-121.test.releng.mdc2.mozilla.com ~]$ /usr/local/bin/pip3 install pyopenssl
Collecting pyopenssl
  Downloading https://files.pythonhosted.org/packages/9e/de/f8342b68fa9e981d348039954657bdf681b2ab93de27443be51865ffa310/pyOpenSSL-19.1.0-py2.py3-none-any.whl (53kB)
     |████████████████████████████████| 61kB 6.2MB/s
Collecting cryptography>=2.8
  Downloading https://files.pythonhosted.org/packages/6b/4a/ce93178469d4460d6b3a5e648fc1a2f426030f3d30a12d7ed4df73d044de/cryptography-2.8-cp34-abi3-macosx_10_6_intel.whl (1.6MB)
     |████████████████████████████████| 1.6MB 27.8MB/s
Collecting six>=1.5.2
  Downloading https://files.pythonhosted.org/packages/65/eb/1f97cb97bfc2390a276969c6fae16075da282f5058082d4cb10c6c5c1dba/six-1.14.0-py2.py3-none-any.whl
Collecting cffi!=1.11.3,>=1.8
  Downloading https://files.pythonhosted.org/packages/d5/61/32b1aa5ef1bf60be4ef679c4aae082a7ceef98517e0e0fde68072c6ef8b6/cffi-1.13.2-cp37-cp37m-macosx_10_6_intel.whl (258kB)
     |████████████████████████████████| 266kB 43.7MB/s
Collecting pycparser
  Downloading https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz (158kB)
     |████████████████████████████████| 163kB 48.0MB/s
Building wheels for collected packages: pycparser
  Building wheel for pycparser (setup.py) ... done
  Created wheel for pycparser: filename=pycparser-2.19-py2.py3-none-any.whl size=111029 sha256=c642f717ddc476bb655ddb51f215b1e5bfd615bb21558f90e03b67a03b786ba5
  Stored in directory: /Users/dhouse/Library/Caches/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511
Successfully built pycparser
Installing collected packages: six, pycparser, cffi, cryptography, pyopenssl
Successfully installed cffi-1.13.2 cryptography-2.8 pycparser-2.19 pyopenssl-19.1.0 six-1.14.0
[dhouse@t-mojave-r7-121.test.releng.mdc2.mozilla.com ~]$ brew upgrade openssl
Warning: openssl 1.1.1d already installed
[dhouse@t-mojave-r7-121.test.releng.mdc2.mozilla.com ~]$ otool -L /usr/local/opt/openssl@1.1/bin/openssl
/usr/local/opt/openssl@1.1/bin/openssl:
	/usr/local/Cellar/openssl@1.1/1.1.1d/lib/libssl.1.1.dylib (compatibility version 1.1.0, current version 1.1.0)
	/usr/local/Cellar/openssl@1.1/1.1.1d/lib/libcrypto.1.1.dylib (compatibility version 1.1.0, current version 1.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
[dhouse@t-mojave-r7-121.test.releng.mdc2.mozilla.com ~]$ find /usr/local/lib/python3.7/site-packages/cryptography/ -name "*.so" | xargs -I{} otool -L {}
/usr/local/lib/python3.7/site-packages/cryptography//hazmat/bindings/_padding.abi3.so:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
/usr/local/lib/python3.7/site-packages/cryptography//hazmat/bindings/_constant_time.abi3.so:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
/usr/local/lib/python3.7/site-packages/cryptography//hazmat/bindings/_openssl.abi3.so:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)

Byron Campen [:bwc]

Reporter

Updated

•

5 years ago

See Also: → https://bugzilla.mozilla.org/show_bug.cgi?id=1340020

Byron Campen [:bwc]

Reporter

Updated

•

4 years ago

See Also: → https://bugzilla.mozilla.org/show_bug.cgi?id=1702323

Andrew Halberstadt [:ahal]

Comment 4

•

4 years ago

Is the only thing needed here uploading a package to internal pypi? Not working today but can tackle next week, please need info if so.

Byron Campen [:bwc]

Reporter

Comment 5

•

4 years ago

It might be enough? We also need the service_identity package now. Just having this working for one platform would be a huge improvement, if that helps expedite.

Andrew Halberstadt [:ahal]

Comment 6

•

4 years ago

Looks like pyOpenSSL==19.1.0 is already on the internal pypi. I can also upload version 20.0.1 if desired.

I uploaded service_identity==18.1.0 while I was there.

Byron Campen [:bwc]

Reporter

Comment 7

•

4 years ago

Just tried this out, we seem to be failing on a dependency (setuptools-rust):

https://treeherder.mozilla.org/jobs?repo=try&revision=dc666cd26f677abfeda14627dda29069cb4f1acf

Jim Mathies [:jimm]

Updated

•

3 years ago

Flags: needinfo?(ahal)

Andrew Halberstadt [:ahal]

Comment 8

•

3 years ago

Also added setuptools-rust==0.12.1.

(And thanks for the needinfo, please needinfo again if there's even more missing or I'll likely miss this again)

Flags: needinfo?(ahal)

Byron Campen [:bwc]

Reporter

Updated

•

3 years ago

Blocks: 1710634

Byron Campen [:bwc]

Reporter

Comment 9

•

3 years ago

Looks like we need yet more deps. Is there a way to run pip against our pypi from a developer machine?

Latest failure on try seems to indicate we need pyasn1-modules. We might also need setuptools>=46.1 and setuptools_scm[toml]>=3.4.3, but it is hard to tell because an intermediate dependency might be a different version than what we have on our pypi. We probably need semantic-version (>=2.6.0 is what pip wants for me), but again it is hard to tell because of differences in version in intermediate dependencies.

Flags: needinfo?(ahal)

Andrew Halberstadt [:ahal]

Comment 10

•

3 years ago

Yeah, you should be able to use pip install --no-index --find-links https://pypi.pub.build.mozilla.org/pub <package>. It will fail on the first package that is missing, but then you could do a normal pip install of that package (ideally in a virtualenv), and then run the first command again to see the next failure. After a few iterations of that you can generate the full list of packages you need.

Flags: needinfo?(ahal)

Byron Campen [:bwc]

Reporter

Comment 11

•

3 years ago

https://treeherder.mozilla.org/jobs?repo=try&revision=10cd04bb49e0d838cc175b483f6aee8204516e83

Byron Campen [:bwc]

Reporter

Comment 12

•

3 years ago

https://treeherder.mozilla.org/#/jobs?repo=try&revision=d4053b9e2c7644962ebc25e30da38f822f7b1850

Byron Campen [:bwc]

Reporter

Comment 13

•

3 years ago

So, after experimenting with a venv, it looks like the only thing we're missing now is pyasn1-modules. The try push in comment 12 ought to be able to pick that up if it is added to the pypi and retriggered, could we give that a go?

Flags: needinfo?(jmaher)

Joel Maher ( :jmaher ) (UTC -8)

Comment 14

•

3 years ago

I have uploaded the module:
https://pypi.pub.build.mozilla.org/pub/pyasn1_modules-0.2.8-py2.py3-none-any.whl

I have also retriggered a few jobs on various platforms to see if this works.

Flags: needinfo?(jmaher)

Byron Campen [:bwc]

Reporter

Updated

•

3 years ago

Blocks: 1731302

Byron Campen [:bwc]

Reporter

Comment 15

•

3 years ago

Thanks! Seems to be working!

Byron Campen [:bwc]

Reporter

Updated

•

3 years ago

Status: NEW → RESOLVED

Closed: 3 years ago

Resolution: --- → FIXED

Bugzilla

Quick Search

Need functioning pyOpenSSL on testers

Categories

(Release Engineering :: General, defect)

Tracking

(Not tracked)

People

(Reporter: bwc, Unassigned)

References

(Blocks 1 open bug)

Details

Crash Data

Security

(public)

User Story

Description

Updated

Comment 1

Comment 2

Updated

Comment 3

Updated

Updated

Comment 4

Comment 5

Comment 6

Comment 7

Updated

Comment 8

Updated

Comment 9

Comment 10

Comment 11

Comment 12

Comment 13

Comment 14

Updated

Comment 15

Updated