Open Bug 1551695 Opened 6 years ago Updated 2 years ago

'schedules' improvements

Categories

(Testing :: General, defect, P3)

Version 3
defect

Tracking

(Not tracked)

People

(Reporter: gbrown, Unassigned)

References

Details

SCHEDULES.inclusive and SCHEDULES.exclusive entries in moz.build files can be a great help in ensuring that the right tests are run for a given changeset, and extra tests are not run.

Can we find more cases where schedules entries would be appropriate?

Idea: review, for each type of test, if a test is updated, is the correct suite run, with other suites excluded?

Idea: review recent landings on autoland; how well did we do / could we improve?

Counter-example: In https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=22dba7347176f6885c4ee3f978aa13a35fc7862e, only mochitest.ini files were updated, but crashtests and xpcshell tests were run :(.

Priority: -- → P3

To confirm, the changes made for mochitest-clipboard subsuite don't overlap with the other test suites that were also run, correct? My impression is that a lot of the test files for non-wpt overlap.

(In reply to Edwin Gao (:egao) from comment #2)

To confirm, the changes made for mochitest-clipboard subsuite don't overlap with the other test suites that were also run, correct?
Correct. In particular, in comment 1 I'm noting that the only file changed was a browser.ini file (a manifest file for mochitests), and that certainly wouldn't affect the browser or any test suites other than mochitest.

My impression is that a lot of the test files for non-wpt overlap.
I'm not sure I understand. I know of a few cases where a test in one suite relies on code in a support file which is also used by a test in another suite, but I think that situation is unusual.

What I observed is that some test files are executed by multiple test suites. Alas, it was just a short in the dark without much in terms of meaningful information to contribute to this bug - sorry about that.

This task could be useful under Reduce redundant tasks.

Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.