Closed Bug 1515653 Opened 6 years ago Closed 6 years ago

TestVerify jobs fail by default first when started from ./mach try fuzzy path

Categories

(Testing :: General, defect, P2)

defect

Tracking

(firefox66 fixed)

RESOLVED FIXED
mozilla66
Tracking Status
firefox66 --- fixed

People

(Reporter: jdescottes, Assigned: gbrown)

References

Details

Attachments

(2 files, 1 obsolete file)

For a while, whenever starting submitting to try using ./mach try fuzzy some/path/to/test.js Will start TV jobs, but they will initially fail for architecture reasons. Once you restart the jobs, they run fine. See for example : https://treeherder.mozilla.org/#/jobs?repo=try&revision=1636fe52304bc10104e8bf7357401a6e6f3d819b
Attached file fuzzy_test_verify_failure.log (deleted) —
Attaching failure logs from the mentioned try build, in case they expire.
Thanks! [task 2018-12-20T13:12:19.680Z] 13:12:19 ERROR - Exception during post-action for download-and-extract: Traceback (most recent call last): [task 2018-12-20T13:12:19.681Z] 13:12:19 ERROR - File "/builds/worker/workspace/mozharness/mozharness/base/script.py", line 2040, in run_action [task 2018-12-20T13:12:19.683Z] 13:12:19 ERROR - method(action, success=success and self.return_code == 0) [task 2018-12-20T13:12:19.685Z] 13:12:19 ERROR - File "/builds/worker/workspace/mozharness/mozharness/mozilla/testing/verify_tools.py", line 49, in find_tests_for_verification [task 2018-12-20T13:12:19.686Z] 13:12:19 ERROR - self.find_modified_tests() [task 2018-12-20T13:12:19.688Z] 13:12:19 ERROR - File "/builds/worker/workspace/mozharness/mozharness/mozilla/testing/per_test_base.py", line 244, in find_modified_tests [task 2018-12-20T13:12:19.689Z] 13:12:19 ERROR - changed_files |= itertools.chain.from_iterable(suite_to_paths.values()) [task 2018-12-20T13:12:19.691Z] 13:12:19 ERROR - TypeError: unsupported operand type(s) for |=: 'set' and 'itertools.chain'
Assignee: nobody → gbrown
Blocks: 1489100
Priority: -- → P2
Attached patch coerce to set (obsolete) (deleted) — Splinter Review
Attachment #9034867 - Flags: review?(mcastelluccio)

:jmaher noted that the same problem might also occur at

https://searchfox.org/mozilla-central/source/taskcluster/taskgraph/util/perfile.py#54

Let me check on that...

Comment on attachment 9034867 [details] [diff] [review] coerce to set Review of attachment 9034867 [details] [diff] [review]: ----------------------------------------------------------------- ::: testing/mozharness/mozharness/mozilla/testing/per_test_base.py @@ +241,4 @@ > changed_files = set() > if os.environ.get('MOZHARNESS_TEST_PATHS', None) is not None: > suite_to_paths = json.loads(os.environ['MOZHARNESS_TEST_PATHS']) > + changed_files |= set(itertools.chain.from_iterable(suite_to_paths.values())) Mmmh, I think is actually wrong (my fault). It should be itertools.chain(suite_to_paths.values()) instead of itertools.chain.from_iterable(suite_to_paths.values()). Nit: Instead of turning it into a set, changed_files.update(itertools.chain(suite_to_paths.values())) should be faster.
Attachment #9034867 - Flags: review?(mcastelluccio)

That doesn't quite work.

For a TV task generated by 'mach try fuzzy <path>', after
suite_to_paths = json.loads(os.environ['MOZHARNESS_TEST_PATHS'])
suite_to_paths might be, for example,
{u'mochitest-clipboard': [u'testing/mochitest/tests/Harness_sanity/test_sanitySimpletest.html']}

Then suite_to_paths.values() == [[u'testing/mochitest/tests/Harness_sanity/test_sanitySimpletest.html']]

And
changed_files.update(itertools.chain(suite_to_paths.values()))
raises: TypeError: unhashable type: 'list'

Whereas
changed_files.update(itertools.chain.from_iterable(suite_to_paths.values()))

produces the correct set in changed_files.

On the other hand, for a TV-bf task generated by a backfill request, the same code sees:

suite_to_paths == {u'clipboard': u'testing/mochitest/tests/Harness_sanity/test_sanity.html'}

Then suite_to_paths.values() == [u'testing/mochitest/tests/Harness_sanity/test_sanitySimpletest.html']

It this case, itertools.chain() will produce the correct value, but itertools.chain.from_iterable() will not.

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=e59086438829c25d3eefe57935164999ca2d2761

I suppose one of those MOZHARNESS_TEST_PATHS is incorrect, but which one? Do you expect:

MOZHARNESS_TEST_PATHS = {'suite': 'test1'}

or

MOZHARNESS_TEST_PATHS = {'suite': ['test1']}

?

Flags: needinfo?(mcastelluccio)

(In reply to Geoff Brown [:gbrown] from comment #7)

That doesn't quite work.

[...]

I suppose one of those MOZHARNESS_TEST_PATHS is incorrect, but which one? Do you expect:

MOZHARNESS_TEST_PATHS = {'suite': 'test1'}

or

MOZHARNESS_TEST_PATHS = {'suite': ['test1']}

?

Yes my bad, so the code was not wrong there and doesn't need to be changed.
The correct one is the one with the list, that is the second one.

So we'll need to fix https://searchfox.org/mozilla-central/rev/c3ebaf6de2d481c262c04bb9657eaf76bf47e2ac/taskcluster/taskgraph/actions/backfill.py#146 to make the value of the dict a list of paths (in this case a list of one path) instead of a path.

Flags: needinfo?(mcastelluccio)

Thanks :marco.

I've used update() and made some cosmetic changes to use the same code in per_test_base.py as in perfile.py. With the additional fix for the backfill format, all's well:

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=31184b838b24da12642aef6dc79e0539eea1319c

Attachment #9034867 - Attachment is obsolete: true
Attachment #9035186 - Flags: review?(mcastelluccio)
Attachment #9035186 - Flags: review?(mcastelluccio) → review+
Pushed by gbrown@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/28c04ca5b35c Fix per-test handling of MOZHARNESS_TEST_PATHS; r=marco
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla66
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: