Closed
Bug 1204942
Opened 9 years ago
Closed 9 years ago
Implement initial autoclassification of intermittents in treeherder.
Categories
(Tree Management :: Treeherder, defect)
Tree Management
Treeherder
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: jgraham, Assigned: jgraham)
References
Details
Attachments
(1 file)
The scope of this bug is:
* A intermittent detector that, given multiple retriggers of the same job of which at least one is green, will mark test failures seen in the other jobs as possible targets for automatic classification.
* An automatic classifier that will take individual test failures and compare them to known failures (typically from the detector above) and mark matches where the test and error message are identical.
Comment 1•9 years ago
|
||
Attachment #8662494 -
Flags: review?(mdoglio)
Updated•9 years ago
|
Attachment #8662494 -
Flags: review?(mdoglio) → review+
Updated•9 years ago
|
Assignee: nobody → james
Status: NEW → ASSIGNED
Comment 3•9 years ago
|
||
Commit pushed to master at https://github.com/mozilla/treeherder
https://github.com/mozilla/treeherder/commit/7b6fa254025a5b0688b6c6c7cbc4c700adb6169f
Bug 1204942 - First cut at autoclassify / intermittent orange detection.
This adds an autoclassify command and a detect_intermittents command.
The former is designed to take an incoming job with an error summary
and look for existing results marked as intermittent that are a close
match for the new result. At present only one matcher is implemented;
this requires an exact match in terms of test name, result and error
message. Matching is also constrained to be based on single lines; it
is anticipated that future iterations may add support for matching on
groups of lines.
The detect_intermittents command is designed to take a group of jobs
running on the same push and with the same build job (i.e. same
testsuite, same chunk, etc.) and look for new intermittents to add to
the database. This currently only looks for test failures where there
is at least one green job and one non-green job.
There is currently no UI for seeing matches or for adding new
prototypical intermittents as match candidates. There is also no
integration with bugzilla; future development should add association
of frequent intermittents with bugs.
Updated•9 years ago
|
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•