STAGE and PROD tests failing last several days with following error 09:04:31 from mozdevice import DeviceManagerADB, DroidADB 09:04:31 ImportError: cannot import name DeviceManagerADB

It looks like this slipped though the cracks in bug 1440714, but that bug should offer some clues how to fix it.

Right, that bug deleted DeviceManagerADB from mozdevice and switched all in-tree consumers of mozdevice to use ADBDevice or similar. I'm not familiar with TPS test: Does this code live outside of mozilla-central? Or is it possibly running pre-bug-1440714 test code with a post-bug-1440714 mozdevice?? Happy to answer questions or help here if I can...

This fell off my radar, patch incoming.

Hm, there's some issue when I try to change the setup.py to list 7.0 or 7.0.1 as the dependency version. I think possibly the dependency lists in mozprofile's setup.py is too involved https://searchfox.org/mozilla-central/source/testing/mozbase/mozprofile/setup.py for my version of python? But I can't seem to get that updated, so maybe someone else can give a better guess as to what's happening (In particular, I get `error: Setup script exited with error in mozrunner setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers` when I change `'mozrunner == 6.15'` to `'mozrunner == 7.0.1'`, or really anything that seems like it would resolve to that (> 6.15, etc)). @gbrown: the TPS python driver is in testing/tps, but it doesn't currently run in the same venv as everything else. I'll upload a patch that can unstick e.g. kitcambridge who was trying to get TPS running, but not something we want to land.

STR: gbrown@mozpad:~/src/testing/tps$ python create_venv.py gbrown gbrown@mozpad:~/src/testing/tps$ source gbrown/bin/activate (gbrown)gbrown@mozpad:~/src/testing/tps$ runtps --help Traceback (most recent call last): File "/home/gbrown/src/testing/tps/gbrown/bin/runtps", line 9, in <module> load_entry_point('tps==0.5', 'console_scripts', 'runtps')() File "/home/gbrown/src/testing/tps/gbrown/local/lib/python2.7/site-packages/pkg_resources.py", line 356, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "/home/gbrown/src/testing/tps/gbrown/local/lib/python2.7/site-packages/pkg_resources.py", line 2431, in load_entry_point return ep.load() File "/home/gbrown/src/testing/tps/gbrown/local/lib/python2.7/site-packages/pkg_resources.py", line 2147, in load ['__name__']) File "/home/gbrown/src/testing/tps/gbrown/local/lib/python2.7/site-packages/tps-0.5-py2.7.egg/tps/__init__.py", line 5, in <module> from . firefoxrunner import TPSFirefoxRunner File "/home/gbrown/src/testing/tps/gbrown/local/lib/python2.7/site-packages/tps-0.5-py2.7.egg/tps/firefoxrunner.py", line 13, in <module> from mozrunner import FirefoxRunner File "/home/gbrown/src/testing/tps/gbrown/local/lib/python2.7/site-packages/mozrunner-6.15-py2.7.egg/mozrunner/__init__.py", line 8, in <module> from .cli import * File "/home/gbrown/src/testing/tps/gbrown/local/lib/python2.7/site-packages/mozrunner-6.15-py2.7.egg/mozrunner/cli.py", line 12, in <module> from .application import get_app_context File "/home/gbrown/src/testing/tps/gbrown/local/lib/python2.7/site-packages/mozrunner-6.15-py2.7.egg/mozrunner/application.py", line 12, in <module> from mozdevice import DeviceManagerADB, DroidADB ImportError: cannot import name DeviceManagerADB

(In reply to Thom Chiovoloni [:tcsc] from comment #4) > Hm, there's some issue when I try to change the setup.py to list 7.0 or > 7.0.1 as the dependency version. I think possibly the dependency lists in > mozprofile's setup.py is too involved > https://searchfox.org/mozilla-central/source/testing/mozbase/mozprofile/ > setup.py for my version of python? But I can't seem to get that updated, so > maybe someone else can give a better guess as to what's happening > > (In particular, I get `error: Setup script exited with error in mozrunner > setup command: 'install_requires' must be a string or list of strings > containing valid project/version requirement specifiers` when I change > `'mozrunner == 6.15'` to `'mozrunner == 7.0.1'`, or really anything that > seems like it would resolve to that (> 6.15, etc)). I get the same error if I try to manually install mozrunner in the venv. I thought pip might be out of date (`pip --version` reports `pip 1.5.6 from /Users/k/tps-venv/lib/python2.7/site-packages (python 2.7)`, and the current version is 10.0.1), running `pip install -U pip` in the venv, then `pip install mozrunner`, still gives me the same error.

I would think a simple mozrunner version update is all that is required: diff --git a/testing/tps/setup.py b/testing/tps/setup.py --- a/testing/tps/setup.py +++ b/testing/tps/setup.py @@ -10,17 +10,17 @@ version = '0.5' deps = ['httplib2 == 0.9.2', 'mozfile == 1.2', 'mozhttpd == 0.7', 'mozinfo == 0.10', 'mozinstall == 1.15', 'mozprocess == 0.26', 'mozprofile == 1.0.0', - 'mozrunner == 6.15', + 'mozrunner == 7.0.1', 'mozversion == 1.5', ] # we only support python 2.6+ right now assert sys.version_info[0] == 2 assert sys.version_info[1] >= 6 setup(name='tps', But as :tcsc reported, that fails: gbrown@mozpad:~/src/testing/tps$ python create_venv.py gbrown Downloading virtualenv 1.11.6 Creating new virtual environment New python executable in gbrown/bin/python Installing setuptools, pip...done. running install running bdist_egg running egg_info writing requirements to tps.egg-info/requires.txt writing tps.egg-info/PKG-INFO writing top-level names to tps.egg-info/top_level.txt writing dependency_links to tps.egg-info/dependency_links.txt writing entry points to tps.egg-info/entry_points.txt reading manifest file 'tps.egg-info/SOURCES.txt' writing manifest file 'tps.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build/bdist.linux-x86_64/egg creating build/bdist.linux-x86_64/egg/tps copying build/lib.linux-x86_64-2.7/tps/phase.py -> build/bdist.linux-x86_64/egg/tps copying build/lib.linux-x86_64-2.7/tps/cli.py -> build/bdist.linux-x86_64/egg/tps copying build/lib.linux-x86_64-2.7/tps/firefoxrunner.py -> build/bdist.linux-x86_64/egg/tps copying build/lib.linux-x86_64-2.7/tps/__init__.py -> build/bdist.linux-x86_64/egg/tps copying build/lib.linux-x86_64-2.7/tps/testrunner.py -> build/bdist.linux-x86_64/egg/tps byte-compiling build/bdist.linux-x86_64/egg/tps/phase.py to phase.pyc byte-compiling build/bdist.linux-x86_64/egg/tps/cli.py to cli.pyc byte-compiling build/bdist.linux-x86_64/egg/tps/firefoxrunner.py to firefoxrunner.pyc byte-compiling build/bdist.linux-x86_64/egg/tps/__init__.py to __init__.pyc byte-compiling build/bdist.linux-x86_64/egg/tps/testrunner.py to testrunner.pyc installing package data to build/bdist.linux-x86_64/egg running install_data copying config/config.json.in -> build/bdist.linux-x86_64/egg/tps creating build/bdist.linux-x86_64/egg/EGG-INFO copying tps.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO copying tps.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying tps.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying tps.egg-info/entry_points.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying tps.egg-info/not-zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO copying tps.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying tps.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO creating 'dist/tps-0.5-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) Processing tps-0.5-py2.7.egg creating /home/gbrown/src/testing/tps/gbrown/lib/python2.7/site-packages/tps-0.5-py2.7.egg Extracting tps-0.5-py2.7.egg to /home/gbrown/src/testing/tps/gbrown/lib/python2.7/site-packages Adding tps 0.5 to easy-install.pth file Installing runtps script to /home/gbrown/src/testing/tps/gbrown/bin Installed /home/gbrown/src/testing/tps/gbrown/lib/python2.7/site-packages/tps-0.5-py2.7.egg Processing dependencies for tps==0.5 Searching for mozversion==1.5 Reading https://pypi.python.org/simple/mozversion/ Best match: mozversion 1.5 Downloading https://files.pythonhosted.org/packages/5c/43/ff7bd757e82e8daa9c3ccd1ff17647f6b6801d154b155db535076023c700/mozversion-1.5.tar.gz#sha256=e9b11e4a46bf7a4a11469ea4589c75f3ba50b34b7801e7edf1a09147af8bf70f Processing mozversion-1.5.tar.gz Writing /tmp/easy_install-M_Mk_R/mozversion-1.5/setup.cfg Running mozversion-1.5/setup.py -q bdist_egg --dist-dir /tmp/easy_install-M_Mk_R/mozversion-1.5/egg-dist-tmp-51io0l Adding mozversion 1.5 to easy-install.pth file Installing mozversion script to /home/gbrown/src/testing/tps/gbrown/bin Installed /home/gbrown/src/testing/tps/gbrown/lib/python2.7/site-packages/mozversion-1.5-py2.7.egg Searching for mozrunner==7.0.1 Reading https://pypi.python.org/simple/mozrunner/ Best match: mozrunner 7.0.1 Downloading https://files.pythonhosted.org/packages/be/d1/90b239e4a43f060e9772a4fba9adf6f8d7a76671771ad7eb239f00af4bd5/mozrunner-7.0.1.tar.gz#sha256=a4e3d532d1d7eb34b6332e0568612ebac3f56da849894005b3dbe68968e5d292 Processing mozrunner-7.0.1.tar.gz Writing /tmp/easy_install-ijBA15/mozrunner-7.0.1/setup.cfg Running mozrunner-7.0.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-ijBA15/mozrunner-7.0.1/egg-dist-tmp-HtNzeI error: Setup script exited with error in mozrunner setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers Traceback (most recent call last): File "create_venv.py", line 194, in <module> main() File "create_venv.py", line 160, in main os.path.join(here, 'setup.py'), 'install']) File "/usr/lib/python2.7/subprocess.py", line 541, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['gbrown/bin/python', '/home/gbrown/src/testing/tps/setup.py', 'install']' returned non-zero exit status 1

On the other hand, I can install mozrunner 7.0.1 manually: gbrown@mozpad:~/xx$ virtualenv gb gbrown@mozpad:~/xx$ source gb/bin/activate (gb) gbrown@mozpad:~/xx$ pip install mozrunner==7.0.1 Collecting mozrunner==7.0.1 Downloading https://files.pythonhosted.org/packages/be/d1/90b239e4a43f060e9772a4fba9adf6f8d7a76671771ad7eb239f00af4bd5/mozrunner-7.0.1.tar.gz Collecting mozdevice>=1.* (from mozrunner==7.0.1) Downloading https://files.pythonhosted.org/packages/49/ea/206bd3e105b8d52fe754c8609babd6548358a8daf766bea7847fe3f18f2c/mozdevice-1.0.0.tar.gz Collecting mozfile==1.* (from mozrunner==7.0.1) Downloading https://files.pythonhosted.org/packages/83/e2/ea5cbdecefd2fd824a836fc5bd16c254a903e1597e9708fab427b4024e0b/mozfile-1.2.tar.gz Collecting mozinfo<1,>=0.7 (from mozrunner==7.0.1) Downloading https://files.pythonhosted.org/packages/db/a3/ef600acad5177fc1dfe822f6645a9153a3a6c72213f061211ba600f935e0/mozinfo-0.10.tar.gz Collecting mozlog==3.* (from mozrunner==7.0.1) Downloading https://files.pythonhosted.org/packages/17/cc/027cef978ec8a630e0c6ff5d5d23a0a54dd35908a778011ae809a9cac10f/mozlog-3.7-py2-none-any.whl (58kB) 100% |████████████████████████████████| 61kB 956kB/s Collecting mozprocess<1,>=0.23 (from mozrunner==7.0.1) Downloading https://files.pythonhosted.org/packages/2f/f8/25c6593b16eda2dc1bea23a0ec92c59158c095981c5d8393ffd2379034bb/mozprocess-0.26-py2-none-any.whl Collecting mozprofile<2,>=1.1.0 (from mozrunner==7.0.1) Downloading https://files.pythonhosted.org/packages/02/10/ed5dec83cd3c7ae0106feeefd4dc403d5c51ff28f49266a176cdb3cc4bb9/mozprofile-1.1.0-py2-none-any.whl Collecting six<2,>=1.10.0 (from mozrunner==7.0.1) Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl Collecting moznetwork>=0.24 (from mozdevice>=1.*->mozrunner==7.0.1) Downloading https://files.pythonhosted.org/packages/d3/bf/69d003109e837f3e5a3d74f44407f3006fb3eb588968246b56ead8a26503/moznetwork-0.27.tar.gz Collecting blessings>=1.3 (from mozlog==3.*->mozrunner==7.0.1) Downloading https://files.pythonhosted.org/packages/81/bd/2629c84d08b9da0252ba8f139a8791b8b7416ed3b7537173e05c35588c2c/blessings-1.6.1-py2-none-any.whl Collecting mozterm (from mozlog==3.*->mozrunner==7.0.1) Downloading https://files.pythonhosted.org/packages/f6/90/aff4ccdae22753369be0a14afbaf0f79f774daf4b24882b12ad540477933/mozterm-0.1.0-py2-none-any.whl Building wheels for collected packages: mozrunner, mozdevice, mozfile, mozinfo, moznetwork Running setup.py bdist_wheel for mozrunner ... done Stored in directory: /home/gbrown/.cache/pip/wheels/af/b9/d9/4f23babe8d570cb8daab6c5da7576495fb6693953bc734130a Running setup.py bdist_wheel for mozdevice ... done Stored in directory: /home/gbrown/.cache/pip/wheels/7e/fb/56/eb1a0a9fa86ff3c244808dfb3e8a14c706ad8c218237460b9d Running setup.py bdist_wheel for mozfile ... done Stored in directory: /home/gbrown/.cache/pip/wheels/b8/87/92/aebf86f8305c60de92ed0694226a3049fd8b72b82d64e27c88 Running setup.py bdist_wheel for mozinfo ... done Stored in directory: /home/gbrown/.cache/pip/wheels/7c/74/af/c04213ac535a149f5ebe9c1d0f216d7eaa14d2f4847c9c8d74 Running setup.py bdist_wheel for moznetwork ... done Stored in directory: /home/gbrown/.cache/pip/wheels/c3/f6/ed/ac0af1e6be733e088b2f70845130a48fb16cbb0ccb5f89db28 Successfully built mozrunner mozdevice mozfile mozinfo moznetwork Installing collected packages: mozfile, blessings, mozterm, six, mozlog, mozinfo, moznetwork, mozprocess, mozdevice, mozprofile, mozrunner Successfully installed blessings-1.6.1 mozdevice-1.0.0 mozfile-1.2 mozinfo-0.10 mozlog-3.7 moznetwork-0.27 mozprocess-0.26 mozprofile-1.1.0 mozrunner-7.0.1 mozterm-0.1.0 six-1.11.0

Hmm, would you mind posting your Python and pip versions? I get this if I try to manually install mozrunner locally: > (tps-venv)k@KC-MBA ~/tps-venv> pip install mozrunner==7.0.1 > Collecting mozrunner==7.0.1 > Using cached https://files.pythonhosted.org/packages/be/d1/90b239e4a43f060e9772a4fba9adf6f8d7a76671771ad7eb239f00af4bd5/mozrunner-7.0.1.tar.gz > Complete output from command python setup.py egg_info: > error in mozrunner setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers > > ---------------------------------------- > Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/8w/nf4nw9vn45b5802570r63hqw0000gn/T/pip-install-cWpQr3/mozrunner/ > (tps-venv)k@KC-MBA ~/tps-venv> pip --version > pip 10.0.1 from /Users/k/tps-venv/lib/python2.7/site-packages/pip (python 2.7) > (tps-venv)k@KC-MBA ~/tps-venv> python --version > Python 2.7.14

(gb) gbrown@mozpad:~/xx$ pip --version pip 10.0.1 from /home/gbrown/xx/gb/local/lib/python2.7/site-packages/pip (python 2.7) (gb) gbrown@mozpad:~/xx$ python --version Python 2.7.12

Aha, this works for me! Does it work for you? Updating VERSION_VIRTUALENV resolved "must be a string or list of strings" but still complained about a conflict in mozprofile versions -- easily addressed.

Comment on attachment 8973033 [details] [diff] [review] update virtualenv version + update mozrunner and mozprofile versions runtps works for me locally now! \o/ Thanks, gbrown!

Comment on attachment 8973033 [details] [diff] [review] update virtualenv version + update mozrunner and mozprofile versions Review of attachment 8973033 [details] [diff] [review]: ----------------------------------------------------------------- It's probably going to continue to be problematic that we pin all of these but this seems fine for now.

Pushed by gbrown@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/8f5a675993ab Update tps package versions; r=tcsc