Closed Bug 1487153 Opened 6 years ago Closed 2 years ago

[moztest] Convert tests from unittests to pytest

Categories

(Testing :: Mozbase, defect, P2)

Version 3
defect

Tracking

(firefox108 fixed)

RESOLVED FIXED
108 Branch
Tracking Status
firefox108 --- fixed

People

(Reporter: davehunt, Assigned: gbrown, Mentored)

References

Details

Attachments

(4 files)

This bug is for converting the moztest unittests to pytest. See bug 1397417 for the motivations and further details.
I have converted the moztest unittests to pytests. I followed the following steps, and got an error - 1. Changed my cd to tests via mozilla-central/testing/mozbase/moztest/tests. 2. Ran the command: $pytest <filename> The file "test.py" gave - "ModuleNotFoundError: No module named 'mozunit'", while "test_resolve.py" gave "ModuleNotFoundError: No module named 'cPickle' " as an error (both in the original and converted files)
(In reply to Sonakshi Saxena from comment #1) > I have converted the moztest unittests to pytests. I followed the following > steps, and got an error - > > 1. Changed my cd to tests via mozilla-central/testing/mozbase/moztest/tests. > 2. Ran the command: $pytest <filename> > > The file "test.py" gave - "ModuleNotFoundError: No module named 'mozunit'", > while "test_resolve.py" gave "ModuleNotFoundError: No module named 'cPickle' > " as an error (both in the original and converted files) You will need to use `mach` to run these tests. Change to the root directory of the repository and try running: > mach python-test testing/mozbase/moztest Please report back your results.
Assignee: nobody → sonakshisaxena1
Status: NEW → ASSIGNED
(In reply to Dave Hunt [:davehunt] ⌚️UTC+1 from comment #2) > (In reply to Sonakshi Saxena from comment #1) > > I have converted the moztest unittests to pytests. I followed the following > > steps, and got an error - > > > > 1. Changed my cd to tests via mozilla-central/testing/mozbase/moztest/tests. > > 2. Ran the command: $pytest <filename> > > > > The file "test.py" gave - "ModuleNotFoundError: No module named 'mozunit'", > > while "test_resolve.py" gave "ModuleNotFoundError: No module named 'cPickle' > > " as an error (both in the original and converted files) > > You will need to use `mach` to run these tests. Change to the root directory > of the repository and try running: > > > mach python-test testing/mozbase/moztest > > Please report back your results. It says - > -bash: mach: command not found I hope this command works well in MacOS too.
(In reply to Sonakshi Saxena from comment #3) > I hope this command works well in MacOS too. It does (I use macOS). Can you try: > ./mach python-test testing/mozbase/moztest Note the preceding `./`
(In reply to Dave Hunt [:davehunt] ⌚️UTC+1 from comment #4) > (In reply to Sonakshi Saxena from comment #3) > > I hope this command works well in MacOS too. > > It does (I use macOS). Can you try: > > > ./mach python-test testing/mozbase/moztest > > Note the preceding `./` Initially, it installed something but then returned the following error - >Error running mach: > ['python-test', 'testing/mozbase/moztest'] The error occurred in code that was called by the mach command. This is either a bug in the called code itself or in the way that mach is calling it. You should consider filing a bug for this issue. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: OSError: [Errno 2] No such file or directory File "/Users/sona/Desktop/mozilla-central/python/mach_commands.py", line 92, in python_test return self.run_python_tests(*args, **kwargs) File "/Users/sona/Desktop/mozilla-central/python/mach_commands.py", line 105, in run_python_tests self.activate_pipenv(pipfile=None, populate=True, python=python) File "/Users/sona/Desktop/mozilla-central/python/mozbuild/mozbuild/base.py", line 764, in activate_pipenv self.virtualenv_manager.activate_pipenv(pipfile, populate, python) File "/Users/sona/Desktop/mozilla-central/python/mozbuild/mozbuild/virtualenv.py", line 601, in activate_pipenv self.virtualenv_root = ensure_venv() File "/Users/sona/Desktop/mozilla-central/python/mozbuild/mozbuild/virtualenv.py", line 575, in ensure_venv venv = get_venv() File "/Users/sona/Desktop/mozilla-central/python/mozbuild/mozbuild/virtualenv.py", line 591, in get_venv env=env).rstrip() File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 216, in check_output process = Popen(stdout=PIPE, *popenargs, **kwargs) File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 394, in __init__ errread, errwrite) File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1047, in _execute_child raise child_exception
It looks like your system is failing to locate pipenv. Could you please paste the full output from running the command? It might also be worthwhile running `./mach bootstrap` to ensure you have any dependencies that you may need. You can select Firefox Desktop Artifact build when prompted, however there's no need to build Firefox.
Attached image Error screenshot (deleted) —
I have attached the full output of the > ./mach python-test testing/mozbase/moztest Also, I didn't get any such choice on typing - > ./mach bootstrap The following is the output of the same - Note on Artifact Mode: Artifact builds download prebuilt C++ components rather than building them locally. Artifact builds are recommended for people working on Firefox or Firefox for Android frontends. They are unsuitable for those working on C++ code. For more information see: https://developer.mozilla.org/en-US/docs/Artifact_builds. Please choose the version of Firefox you want to build: 1. Firefox for Desktop Artifact Mode 2. Firefox for Desktop 3. Firefox for Android Artifact Mode 4. Firefox for Android Your choice:
By "Firefox Desktop Artifact" I meant "Firefox for Desktop Artifact Mode".
Oh, okay! What should I be doing now? Should I try to build it? Will that solve the problem?
No, you don't need to build. Did you complete `./mach bootstrap` with the selection mentioned? Has it made any difference to the `./mach python-test testing/mozbase/moztest` command?
Yes, I completed `./mach bootstrap` but it made no difference to the `./mach python-test testing/mozbase/moztest` command.
Odd, it looks we can't find `pipenv` in your environment. It should be using the version at /third_party/python/pipenv, but either there is something in your environment that's causing this not to work or there's a bug in the virtualenv creation somewhere.
Is there any solution to this? I had already converted the 'unittests' to 'pytests', so how should I test them?
(In reply to Sonakshi Saxena from comment #13) > Is there any solution to this? > I had already converted the 'unittests' to 'pytests', so how should I test > them? We need to understand why `./mach python-test` is not working, as this is how you will need to test these changes. Have you made any changes other than within the `testing/mozbase/moztest` directory? Your console output is lacking some details regarding setting up the virtual environments. Could you perhaps try running `./mach clobber` followed by `./mach python-test testing/mozbase/moztest` and attach the full console output again?
Also make sure you are on the latest checkout (in case this was a problem that got fixed already): $ hg pull -u
Attached file Output.pdf (deleted) —
No, I have not made any changes. I also 'checkout' as per guided. './mach clobber' did end up installing 'pipenv' but it is still giving an error.
Please note the ValueError due to an unknown locale UTF-8. Maybe that is the problem, or a part of it.
Should I try adding this - > export LC_ALL=en_US.UTF-8 > export LANG=en_US.UTF-8
This looks more as a locale, yes. UTF-8 is only an encoding.
Attached file Output.pdf (deleted) —
I think I have fixed it :D I added the following to .bash_profile - > export LANG="it_IT.UTF-8" > export LC_COLLATE="it_IT.UTF-8" > export LC_CTYPE="it_IT.UTF-8" > export LC_MESSAGES="it_IT.UTF-8" > export LC_MONETARY="it_IT.UTF-8" > export LC_NUMERIC="it_IT.UTF-8" > export LC_TIME="it_IT.UTF-8" > export LC_ALL= Please check the attachment once, just to be sure.
That looks great. Btw. you don't have to attach PDF files, but can directly paste the content when you select `paste as attachment` on the file upload page. Text attachments are way better accessible.
I am sorry, I didn't know about that, since I am new to Bugzilla. Also, I am facing a new problem now. I tried to test my converted files but I am getting the following error - > No build detected, test metadata may be incomplete. > No handlers could be found for logger "mozbuild.frontend.reader" > 0:08.29 TEST-UNEXPECTED-FAIL | No tests collected (Not in PYTHON_UNITTEST_MANIFESTS?) Can you help me with this?
That means that |mach python-tests| couldn't find any tests to run. Common causes are mistyping the path, forgetting to add your test to the manifest or forgetting to add your manifest to a moz.build file. Since you aren't adding any new tests, please double check the path you typed exactly matches the one :davehunt pasted. Also make sure you are running from the root of the repository.
I have solved the above issue. Thank you all for helping me out! :) I'll make the patches soon! Please review them, once done.
I need help in creating/submitting patches since I am new to Mercurial. (although I have used git before)
(In reply to Sonakshi Saxena from comment #25) > I need help in creating/submitting patches since I am new to Mercurial. > (although I have used git before) We're using Phabricator for reviewing patches. Please read through the user guide at https://moz-conduit.readthedocs.io/ to set up your account, install the required software, and submit your patch.
Okay. Thankyou.
Priority: -- → P2

Sonakshi: If you wish to finish up this bug, please reassign it to yourself. If you need help with Phabricator or anything related let me know. I'll be glad to help.

Assignee: sonakshisaxena1 → nobody
Status: ASSIGNED → NEW

I'm no longer a suitable mentor for this bug, and unsure if it's still valid. What do you think :ahal?

Mentor: dave.hunt
Flags: needinfo?(ahal)

Yeah, it's still valid. I can take over mentorship :)

Mentor: ahal
Flags: needinfo?(ahal)
Severity: normal → S3
Assignee: nobody → gbrown
Status: NEW → ASSIGNED
Pushed by gbrown@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/2eef0bd36cd5 Convert moztest tests to pytest; r=ahal
Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → 108 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: