Closed Bug 1507635 Opened 6 years ago Closed 6 years ago

Running reftest/crashtest tests in isolation from 'mach try fuzzy' doesn't work

Categories

(Testing :: General, defect, P3)

Product:
Testing
Component:
General
Type:
defect

Tracking

(firefox65 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla65
Tracking Flags:
Tracking Status
firefox65 --- fixed

People

(Reporter: marco, Assigned: marco)

References

(Blocks 1 open bug)

Details

Attachments

(4 files)

reftest.tar.xz
(deleted), application/x-xz
Details
When running specific tests on try, don't use the 'tests' attribute from the suite
(deleted), patch
ahal
: review+
Details | Diff | Splinter Review
Log reftest.manifests and reftest.tests when there is an error with them
(deleted), patch
ahal
: review+
Details | Diff | Splinter Review
reftest paths should be relative to abs_reftest_dir/tests
(deleted), patch
ahal
: review+
Details | Diff | Splinter Review
Attached file reftest.tar.xz (deleted) —
./mach try fuzzy layout/reftests/css-ruby/nested-ruby-1.html layout/reftests/w3c-css/submitted/counter-styles-3/disclosure-styles.html The failure is: > runreftest.py: error: unrecognized arguments: -- tests/reftest/tests/layout/reftests/reftest.list
After bug 1507767 is fixed, crashtests can be scheduled but they fail with a similar error: > runreftest.py: error: unrecognized arguments: -- tests/reftest/tests/testing/crashtest/crashtests.list
Priority: -- → P3
Summary: Running reftest tests in isolation doesn't work → Running reftest tests in isolation from 'mach try fuzzy' doesn't work
Summary: Running reftest tests in isolation from 'mach try fuzzy' doesn't work → Running reftest/crashtest tests in isolation from 'mach try fuzzy' doesn't work
Assignee: nobody → mcastelluccio
Status: NEW → ASSIGNED
This is the first step to make this work. After applying this, reftests start but fail very early with: > ERROR - REFTEST ERROR | Exactly one of reftest.manifests or reftest.tests must be specified.
Attachment #9026096 - Flags: review?(ahal)
Comment on attachment 9026134 [details] [diff] [review] Log reftest.manifests and reftest.tests when there is an error with them Review of attachment 9026134 [details] [diff] [review]: ----------------------------------------------------------------- ::: layout/tools/reftest/reftest.jsm @@ +363,4 @@ > > if ((testList && manifests) || !(testList || manifests)) { > logger.error("Exactly one of reftest.manifests or reftest.tests must be specified."); > + logger.debug("reftest.manifests is: " + manifests); Did you mean to make this debug?
Attachment #9026134 - Flags: review?(ahal) → review+
Comment on attachment 9026136 [details] [diff] [review] reftest paths should be relative to abs_reftest_dir/tests Review of attachment 9026136 [details] [diff] [review]: ----------------------------------------------------------------- Thanks for catching and fixing this!
Attachment #9026136 - Flags: review?(ahal) → review+
Attachment #9026096 - Flags: review?(ahal) → review+
Pushed by mcastelluccio@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/ab36ef2d6d00 When running specific tests on try, don't use the 'tests' attribute from the suite. r=ahal https://hg.mozilla.org/integration/mozilla-inbound/rev/8b24e89c77f1 Log reftest.manifests and reftest.tests when there is an error with them. r=ahal https://hg.mozilla.org/integration/mozilla-inbound/rev/da7bf8203ec1 reftest paths should be relative to abs_reftest_dir/tests. r=ahal
https://hg.mozilla.org/mozilla-central/rev/ab36ef2d6d00 https://hg.mozilla.org/mozilla-central/rev/8b24e89c77f1 https://hg.mozilla.org/mozilla-central/rev/da7bf8203ec1
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
status-firefox65: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla65
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: